Jump to content

Problems with importing images


franky68
 Share

Recommended Posts

Hello,

 

I am a bit worried now...

 

I just got the error about the timeout while importing images. After that one I closed Daminion and started it again. Instead of importing a selection of folders at once now I tried to import them one by one.

 

So I stated with one folder, imported it and after it finished I imported the next folder. Now I noticed that the counters for the folders (in the Catalog tags window) were not updated. So I restarted Daminion again to see if it would show the actual numbers.

 

After the restarted I noticed 6 broken images right on top of the catalog. I checked and noticed that they were in one of the folders I previously imported and of which I am sure they were there, correctly imported and the thumbs displayed. Some of exactly those images now are reported as broken.

 

Of course I checked the folder which I imported them from and found they are gone from the folder as well. Checking the complete catalog I found a total of 262 images that are *gone*. They all have been imported into the catalog and now show as missing image - and they all are gone from the disk as well.

 

Yes, I do have backups and so I will be able to restore them - and yes, I know Daminion (the server in particular) is still beta and so I am using it on my own risk. But I guess you can imagion I am pretty <censored> - I did not expect this even from a beta... :ireful3:

Link to comment
Share on other sites

Could you please describe:

- on what PC was installed the Daminion Server

- on what PC was launched the Daminion Client

- and on what PC (and folder) was stored your importing files

 

Please tell me - did you check an option "Upload files to the Server" option before importing?

 

Could you please send me your log folder - we need to take a look at the issue from your previous post (timeout issue) in details.

Link to comment
Share on other sites

Both (client and server) were running/installed on my desktop. This also is the PC which holds the drive with my images.

 

Its a Q9550 with 4GB RAM

 

The images are/were stored in

D:\Fotos\Öffentlich\2011\2011.10\

 

And no the "upload" option was unchecked. The deleted files are not in the recycle bin nor in the daminion data directory (C:\ProgramData\Daminion Software\Daminion Server\Data).

 

I just double checked and located three series of images that are gone. I am not absolutely sure about the total number (262 is what Daminion shows for broken files) but there are definately more than 200 images gone that I *know* have still been there yesterday. Those from the one folder definately were there until the import and i suppose the others were deleted on the previous import that stalled.

 

I am sending you the logfile now - and will be offline for a while.

Link to comment
Share on other sites

Those from the one folder definately were there until the import and i suppose the others were deleted on the previous import that stalled.

 

Daminion will not change or delete files during the importing - it only generate previews and extract the metadata. Did you catalog the same folder/files on other DAM software or any chance that the folders were accidentally moved (by mouse for ex) to other location?

 

 

Did you check out/in the files that were disappeared?

 

What was the format of the deleted files? NEF, JPEG or NEF + JPEG?

Link to comment
Share on other sites

We've located the problem... very sad, but there are some combination of factors when the Daminion Server 0.8.7.443 can delete files. :sorry2::unknw:

 

The problem is specific to the Daminion Server 0.8.7.443 version only and appears if:

  • "Upload to the Server" option is checked off
  • you try to import the files that were already imported
  • the importing files are located on the Server PC

 

Please give our apologizes, we'll release the patch during an hour or so. I'll send the direct link to a new version for all users who received an invitation to test the Daminion Server 0.8.7.443.

 

Please use the Daminion Server with test media collections only, until we release the Release version.

Link to comment
Share on other sites

Daminion will not change or delete files during the importing

 

very sad, but there are some combination of factors when the Daminion Server 0.8.7.443 can delete files.

 

Forgive me if I ask but your first statement is what I would expect. A DAM should never and under no circumstances delete any images unless the user explicitly tells the application to do this (Shift-DEL) but definitely NOT at its own whim.

 

Now an unlink or delete command just does not slip in, it has to be coded into the application. I do not see any reason to have a command like that in the import section. What other functions have instructions to unlink files?

 

The problem is specific to the Daminion Server 0.8.7.443 version only and appears if:

  • "Upload to the Server" option is checked off
  • you try to import the files that were already imported
  • the importing files are located on the Server PC

 

Well #3 is always met along with #1 and #1 will be the normal situation for most users. So were basically down to #2 which in my case was probably caused by the timeout in the previous import. But its still a pretty standard situation - when filed are added to the directory I'd re-import the whole folder instead of handpicking the newly added files. So its not exactly some kind of exotic race condition...

 

Please use the Daminion Server with test media collections only, until we release the Release version.

 

Thats easily said...

 

Until a week ago I only played with a couple of isolated files that but you don't get any "reallife" bugs or problems when just playing with a few files. What you need to find them is to do real work with a bigger selection of images and handle them like you normally would.

 

Since I started this I found quite a few bugs I would not have encountered otherwise. I know I am working with a beta software so I am regularly checking the integrity of my images (and already found a few corruptions too) to make sure I am not loosing files. But I have to admit I was not prepared for (or expected) an import deleting more than half of the images :/

Link to comment
Share on other sites

Please update your Daminion Server to 0.8.7.445 version

 

This is a minor update for Daminion Standalone (Client) version.

 

I knew I shouldn't but for some reason and since I had nothing more to loose I felt brave today and upgraded to those versions.

 

1. Observation: After updating client and server I noticed I could not connect to the shared catalog anymore. Checking the server manager told me that the server would be running tho. I decided to reboot anyway just to be sure.

 

2. Observation: After rebooting the server is not started anymore but is listed as stopped. I was able to manually start the server tho.

 

3. Observation: Now that the server was not only shown as running but really was I was finally able to connect to the catalog. The next shock was that now every image is listed as "missing". You probably can imagion my shock after the earlier experience. But checking the actual folders ensured me that the files were still existent and only Daminion (for whatever reason) things they aint.

 

4. Observation: To have at least one image in the catalog I made a copy of an image and tried to import it into the catalog. Neither the counters (Last imported) was updated nor did I get a thumbnail so obviously it was not imported. Since it was only a copy and I could live with loosing it I bravely tried to import it a second time. The good news: The physical image was not deleted this time. The bad news: The image is still not in the catalog. I tested this on the Desktop (=server) with and without uploading the image.

 

Later I tried the same from the laptop (with uploading of course) and the image was correctly added to the catalog and also visible on the server.

 

Forgive me the question but after all this (you might understand that I'm a bit tense) I start to wonder if server releases are actually tested before they are made public. This is becoming somewhat scary...

Link to comment
Share on other sites

Hi Franky;

 

It may have been helpful if Murat had more clearly indicated that this was an Alpha release, and what to expect from it in terms of stability.

 

I found the .445 release to be quite stable on my system over several hours of testing.

 

Having done a tiny bit of application development, I think I understand the need for outside testing at an early (alpha) stage, as well as beta testing.

 

As a developer I know exactly what I want the software to do, and how to make it do that. I have a specific configuration of hardware and software. I try to test as well as I can, but there are thousands of hardware/ software configuration out there, to say nothing of users who will interact with the system in ways I would never expect. You have to get your software/application out of the controlled development environment to find out how it fares with non-developer users.

 

To look at it another way, in the first few weeks of use of almost any software, I can almost always either break it, or totally botch up the data in ways that I would not be able to repeat after three months of use and familiarity. That is also part of the need for outside testing.

 

Hang in there, the team will come through. Just recognize that alpha software comes with a lot of warts. If you don't want to deal with those, you may want to wait for a beta.

 

Peter

Link to comment
Share on other sites

Frank, I appreciate for the great feedback(s)!

 

The #2, #3, #4 points are derived from the one issue - a new Daminion Server installation reset your "Log On as" option for your DaminionServerService.

 

Please start the Services window, and check what user is belonged to DaminionServerService?

 

You need to change it to a user that has rights to access/write to remote folders/files.

 

In your case Daminion contains a lot of links from the previous version, and now a new installation changed the default user (Log On As) for the DaminionServerService, and Daminion will be able to access them and display the missing links.

 

Please change it and Daminion Server will be able to see your files.

 

We'll include an ability to install the Server with a "power" user rights on the next product update.

Link to comment
Share on other sites

The #2, #3, #4 points are derived from the one issue - a new Daminion Server installation reset your "Log On as" option for your DaminionServerService.

 

Hello Murat,

 

youre correct - the installation replaced the configured user with a default one who did not have access to the directories. I missed to check that and after fixing the user the images in the catalog were back.

 

So my apologies for the last post. False alarm here and I should have checked the user in the first place. I was still a bit cranky about loosing my images, in particular since I discovered that (for entirely my fault) a good few of them weren't backed up :/ So while kicking my own ass and double checking my backup strategy I still wonder how the filedelete slipped into the importer...

 

Anyway this would have been much less of an issue if my backup drive didn't ran out of space unnoticed by me :dash2:

 

BTW: This forum is lacking an offtopic section for stuff not directly related to Daminion. Like suggesting sections, whining about too small backup drives etc. ;)

Link to comment
Share on other sites

BTW: This forum is lacking an offtopic section for stuff not directly related to Daminion. Like suggesting sections, whining about too small backup drives etc. ;)

 

 

Could you please stop any actions related to copying/moving/saving the files/documents and download a small utility named R-Studio. As you know even if files were deleted from your hard drive they can be easily recovered if you didn't overwrite them with a new info.

 

I can send you link, please let me know.

 

http://www.data-recovery-software.net/Data_Recovery_Download.shtml

Link to comment
Share on other sites

Hmmm.... after scanning my datadrive for about 4 hours it finally recovered a bunch of 1kB NEF files. They are a little too small to be any good. But then again the drive is pretty packed so I am afraid most data has been overwritten already anyway.

 

Thanks for the suggestion tho.

Link to comment
Share on other sites

Yes, there are plenty of recovering tools. Unfortunately they all have one thing in common: Once you did not react fast enough and waited to use them until the next day while the system was actively using that drive - which is pretty packed so there is little chance but to use exactly those sectors - youre out of luck ;)

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...