Paul Barrett Posted January 28, 2017 Report Share Posted January 28, 2017 I was wondering..... Does anyone use Daminion as a means of cataloguing and tagging documents such as bill, receipts, legal documents etc? - Paul Quote Link to comment Share on other sites More sharing options...
Uwe Posted January 28, 2017 Report Share Posted January 28, 2017 Hello, yes I do it. I scan the documents and save the scan as PDF. Regards, Uwe Quote Link to comment Share on other sites More sharing options...
Daria Kotilainen Posted January 30, 2017 Report Share Posted January 30, 2017 Hello Paul, I do exactly the same as Uwe does: scan the documents, save as PDF and add to the catalog. Best regards, Daria Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted January 30, 2017 Author Report Share Posted January 30, 2017 OK so I have created the new server catalog and added the files from the appropriate directory. But I am having issues getting the import to run properly. It keeps stalling. The folders contain 878 files according to Synology File Station. Daminion is only seeing 623 and every time I do a rescan it gets stuck on item 133. It occurs to me that I might have some unsupported file types buried somewhere in the 132 folders. Are there any unsupported file types? Regards Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted January 30, 2017 Author Report Share Posted January 30, 2017 The good news is that this time the rescan report is working (it wasn't in previous attempts) The highlighted actions in the screenshot seem to appear very often. What do they mean please? Regards Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted January 30, 2017 Author Report Share Posted January 30, 2017 75 minutes later it's still stuck at the same point so I have cancelled the rescan. I don't know what to do next. How do I do an automated compare between the catalog folders and contents of the NAS folders? Why is it stuck reading the root of the shared drive folder? It's the folder it appears to be stuck on a file - Paul Quote Link to comment Share on other sites More sharing options...
lintujuh Posted January 31, 2017 Report Share Posted January 31, 2017 Hi Paul, One way of doing the comparison of the contents would be 1) in Daminion – display all files in your catalog – select all (<ctrl>-A) – copy the full path names (Edit > Copy Names with FUll Path, <ctrl><shift>-c) – paste the contents of the clipboard into a text file – sort the text file 2) in Windows command prompt – run command dir /b /s dir > dirlist.txt – sort the output file 3) compare the files (Note the file #2 contains also the directories, which are not present in the Daminion listing) -Juha 1 Quote Link to comment Share on other sites More sharing options...
Daria Kotilainen Posted January 31, 2017 Report Share Posted January 31, 2017 The good news is that this time the rescan report is working (it wasn't in previous attempts) The highlighted actions in the screenshot seem to appear very often. What do they mean please? Regards Paul Hello Paul, the highlighted actions are the info about the folder being rescanned right now (or folders that have already been rescanned). Daminion "mentions" a folder and then checks its contents file by file. Can you please scroll down the rescan report and check what file was the last one when the rescan stuck and what was the status of this file. Daminion adds files one by one as they are in the original folder, so you can try to locate the culprit and import this file separetely. Regards, Daria Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted January 31, 2017 Author Report Share Posted January 31, 2017 Hello Paul, the highlighted actions are the info about the folder being rescanned right now (or folders that have already been rescanned). Daminion "mentions" a folder and then checks its contents file by file. Can you please scroll down the rescan report and check what file was the last one when the rescan stuck and what was the status of this file. Daminion adds files one by one as they are in the original folder, so you can try to locate the culprit and import this file separetely. Regards, Daria One of the things that I should have anticipated was that when you run the report it only includes the activities since the previous report. Thus, scrolling down is limited. I have just restarted a rescan and will leave it running for several hours before I look at the rescan report. - Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted January 31, 2017 Author Report Share Posted January 31, 2017 OK so I started a folder rescan at 09:03 and left it to run. I came back to it just over 4 hours later. Once again it was stuck on an item (A in image below) but as the app does not tell me which file it is currently processing I have no way of investigating the problem. I right clicked the spinning icon and ran the latest rescan report, expecting to see 4 hours' worth of entries but all that was present were the entries for the last 60 seconds or so (B). Is that expected? Help! How do I fix this please? - Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 3, 2017 Author Report Share Posted February 3, 2017 OK so I started a folder rescan at 09:03 and left it to run. I came back to it just over 4 hours later. Once again it was stuck on an item (A in image below) but as the app does not tell me which file it is currently processing I have no way of investigating the problem. I right clicked the spinning icon and ran the latest rescan report, expecting to see 4 hours' worth of entries but all that was present were the entries for the last 60 seconds or so (B). Is that expected? Help! How do I fix this please? - Paul So, another 6 hours later, still no progress had been made. I killed it. Then, today, looking at the catalog for another reason i looked at the Saved Searches and saw that of the 629 documents it contains, 626 are unsynced. So a total of 10 hours processing synced precisely THREE documents. HELLLLLPPPPPP! :) - Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 3, 2017 Author Report Share Posted February 3, 2017 Now that's interesting. I could not get autosync to complete right on these files either. But when I selected them all and did 'Write Metadata to Files' it worked. - Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 3, 2017 Author Report Share Posted February 3, 2017 Latest Update: Process still hangs I have created a New Catalog and tried the following: Is there a file in the root that is causing the problem? No, even with the root clear of files, the process hangs at 450 out of 632 files Is there a file buried in a subfolder that the system can't handle? Tried importing a folder at a time. All attempts resulted in the [process hanging but this time NO files were imported at all. Is it a permissions issue? (Unlikely given that I have been able to import 450 images). Only thing I could find was that the daminion server account did not have Advanced permission in Windows. So I applied them. It made no difference. Is there something inherent in the parent folder \public that is preventing the import?. I copied the entire set of folders into the \photos directory and tried to import them into my main photo catalog. The process spun but did not import a single file Is there an issue with the NAS folder? I moved the entire set of folders down to my PC. One file could not be transferred because it was locked by daminion server. I deleted it in File Station and tried again.no success Then I did an add files. copying them from the PC to the NAS. Import Froze again Then I decided to try a local catalog on my PC's local drive. That looked as though it was going to work but stalled at 590 files. up from the 450 earlier So I deleted some folders and tried again. Now it's stuck at zero files I am all out of ideas.............. Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 4, 2017 Author Report Share Posted February 4, 2017 Latest Update: I decided to go through the folders one-by-one to see if I could find anything out of the ordinary. I came across this one, which looked odd and since it was not an important files I decided to junk it: Then I emptied the folder and re-ran the add files command. The results are shown below: A: The filesystem reports 895 files in total B: Daminion reports it has imported 859 or 97.7%. That percentage equated to 879 in total, which is 16 less than the file system gives C: Daminion reports it has processed 740 of 740? Process is stuck on 20, whatever that means, but there are definitely 740 items in the catalog. So I am 147 files short. Items created today are importing OK. The problem appears to be with existing items, It loos like I will have do an inventory of both systems to find out the deltas. - Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 4, 2017 Author Report Share Posted February 4, 2017 Hi Paul, One way of doing the comparison of the contents would be 1) in Daminion – display all files in your catalog – select all (<ctrl>-A) – copy the full path names (Edit > Copy Names with FUll Path, <ctrl><shift>-c) – paste the contents of the clipboard into a text file – sort the text file 2) in Windows command prompt – run command dir /b /s dir > dirlist.txt – sort the output file 3) compare the files (Note the file #2 contains also the directories, which are not present in the Daminion listing) -Juha This approach worked reasonably well for me. It allowed me to identify some issues. 1. There were many Damionion thumbnail files in the folder. None of my current catalogs point to that location so I moved them out 2. There was a file with a . imn it's name which might have caused a problem so I renamed that 3. There was a file with the extension .jpg_original. I deleted that 4.There were several .exe and .msi files. Bang! Gone. The list of files on the server was therefore reduced to 854. I emptied the catalog and tried again. The import wizard found only 837. Pressed Import. and got: Last time this happened i was asked to provide the details. Here they are: Error.txt Daria / Anybody - any more suggestion please? - Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 4, 2017 Author Report Share Posted February 4, 2017 To rule out it being an issue with the PC, I set up a brand new local catalog on my laptop and imported the files there. Or tried to. Same problem. It gets stuck. Both PCs are on the latest build (1582) If ONLY the import routine would display the name and path of the file currently being processed I could probably get to the bottom of this. As it is, it's like looking for a needle in a haystack. :( - Paul Quote Link to comment Share on other sites More sharing options...
Julek Posted February 5, 2017 Report Share Posted February 5, 2017 Hi Paul, Something to look at..check your computer power settings and see if the computer is putting your drives to sleep after a while. I had this issue with syncing files that are on external Western Digital drives. (In their case, the firmware in the drives force a spin down after a set amount of time) What happened was the drives were spinning down, thus Daminion couldnt write updates to the files/drives If you or anyone has this problem,check out KeepAliveHD. Its a little app that writes a txt doc to any drive you pick (you can choose more then one drive at the same time) By it writing a txt document every few seconds/minutes <You pick) it keeps the drive awake and Daminion can do its thing. You can immediately have KeepAliveHD delete the txt file, so you dont have a bunch of silly files piling up anywhere. I'm no geek, however, given it sounds like the hassle I was going thru, so wanted to put this out there. If its an internal drive, check windows power settings. Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 5, 2017 Author Report Share Posted February 5, 2017 Hi Paul, Something to look at..check your computer power settings and see if the computer is putting your drives to sleep after a while. The hard drives are on a NAS drive so that's not an issue. I have my PC set to not sleep. So connectivity is not the issue, but thanks for the thought :) - Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 5, 2017 Author Report Share Posted February 5, 2017 My latest test was to take the NAS drive out of the equation by downloading the files onto my local hard disc and creating a brand new local catalog . I added the files and waited. I got the exact same behaviour. At 92.5% the process stalled with exactly the same number of files processed. I assume it's stuck on the same file but will I ever find out which one? If you've had the stamina to read this far in this thread I am impressed. Thank you. As I am sure you are aware by now I have no way of knowing which file is causing the problems as Daminion does not tell you which file is being processed. There's a log file of what's BEEN processed, but which file is CURRENTLY being processed? No. And, anyway, the log file is not even an option because, in it's current stuck state, I can't right click the progress box to access the log file option. I have tried to deduce the current file by comparing the list of directory contents with the last file successfully processed, but since consecutive files in the import sequence seem to flip between directories I have to assume the processing sequence is based on something other than 'by folder, by file'. I'm off to dance naked in the woods by moonlight whilst summoning the spirits of my ancestors, It stands as much chance of success as everything else I have tried! :) - Paul Quote Link to comment Share on other sites More sharing options...
Len Posted February 6, 2017 Report Share Posted February 6, 2017 Would it be possible to import in smaller chunks until you find the chunk that causing you problems, then split that one into smaller bits until you find the faulty file ( the one that causing the Problem. Basic law of Fault find is to eliminate the good bits until all that's left is the bad bit. 1 Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 6, 2017 Author Report Share Posted February 6, 2017 Would it be possible to import in smaller chunks until you find the chunk that causing you problems, then split that one into smaller bits until you find the faulty file ( the one that causing the Problem. Basic law of Fault find is to eliminate the good bits until all that's left is the bad bit. Yep, tried that already. No matter how small a chunk I import it hangs,:( But thanks for the thought. :) - Paul Quote Link to comment Share on other sites More sharing options...
Paul Barrett Posted February 8, 2017 Author Report Share Posted February 8, 2017 Would it be possible to import in smaller chunks until you find the chunk that causing you problems, then split that one into smaller bits until you find the faulty file ( the one that causing the Problem. Basic law of Fault find is to eliminate the good bits until all that's left is the bad bit. In the end, Lennard, I was left with no option but to use the smallest chunk possible, individual files! I had a feeling that the trouble was coming from the files in a specific set of folders. To spare myself from navigation nightmares I copied the contents of the sub folders out into a flat folder in a separate location I then stepped through the files one-by-one, adding them to a new local catalog, As each file was added I rescanned the folder I repeated these steps until I found one that caused Daminion to crash, I copied the dodgy file to a quarantine folder I repeated this for the remaining files and found another Both files were PDFs. I moved them out of the main library, Then I rescanned the specific subfolders and they scanned OK So, next I rescanned the entire library. SUCCESS The 'faulty' files in the quarantine were then opened in various apps to see if they too had a problem. All the PDF readers and editors had no problems with the files It appears therefore that there is some characteristic of the files that Daminion can't handle, to such a degree that instead of reporting an error, Daminion simply freezes and has to be killed using Task Manager This was a tedious manual process. I have reported it all to Daminion and suggested that: The log files are not much use because they only contain details of processed and skipped files. We need an errors log that is real time, like the one you get if you try to move a file that is open in another app, so it's in the user's face. We need the progress bar to show the path and file currently being processed so that if Daminion freezes we know which file is responsible. Daminion needs to handle these situations more elegantly and not freeze when something minor happens with a file. I must have used Task Manager to kill a frozen Daminion session at least 20 times in the last two days, which is unacceptable. So far the Synology file system has been robust enough to protect the integrity of the data so that I have not lost a single file but we;re tempting fate. I have had, on several occasion, to dive into Synology File Station to find missing folders that Daminion had reported as missing Daria was going to talk to Alex today. I wonder what the outcome was? - Paul 1 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.