network file server (NAS) support?

[Per-BKWine]

Starting to test Daminion.

 

Does Daminion support image files stored on a NAS?

 

I am working with a local catalogue but all image files are on a NAS on my local network.

 

It seems to read the files relatively OK (although I seem to get quite a lot cataloguing errors/incomplete cataloguings).

 

However, when I try to embed the metadata to the files it seems to stall. I tested embedding data to three files and after 30 minutes it was still stalled on the first image.

 

If I embedded on a locally stored file it was almost instantaneous.

 

If Daminion does support files on a NAS, then why does embedding the metadata take so long? More than half an hour for one file is not reasonable.

 

(The metadata is embedded in the file, not in a sidecar file. This is necessary since it is the way the people I deliver files to require it. They do not accept sidecar files. Embeddeing in the file is also in principle a better option that sidecar files, since there is no risk that the metadata gets separated from the file.)

[Murat]

Starting to test Daminion.

Does Daminion support image files stored on a NAS?

I am working with a local catalogue but all image files are on a NAS on my local network.

 

Yep. Daminion allows you to import files from external network locations (like NAS, SAN or shared folders) by references without copying the actual files. You need check the first option in the Import window:

 

It seems to read the files relatively OK (although I seem to get quite a lot cataloguing errors/incomplete cataloguings).

 

Could you please explain what do you mean by a lot cataloguing errors/incomplete cataloguings?

 

However, when I try to embed the metadata to the files it seems to stall. I tested embedding data to three files and after 30 minutes it was still stalled on the first image.

If I embedded on a locally stored file it was almost instantaneous.

 

 

Metadata updating speed depends from several factors:

- Location of your files, in case of remote locations - Daminion should update your files using the slow network connection

- Image format and Metadata Writing settings. RAW images will be updated slowly if the 3rd option was selected as it shown below:

 

 

- File Size

- Computer configuration (CPU, RAM, system loading) and the network speed (1Gb/s is recommended for comfortable work)

- Local or Server catalog. After reaching 100k files per catalog Daminion Server catalogs work much faster than local catalogs

- Program errors. Usually we quickly fix them.

 

 

If Daminion does support files on a NAS, then why does embedding the metadata take so long? More than half an hour for one file is not reasonable.

 

 

(The metadata is embedded in the file, not in a sidecar file. This is necessary since it is the way the people I deliver files to require it. They do not accept sidecar files. Embeddeing in the file is also in principle a better option that sidecar files, since there is no risk that the metadata gets separated from the file.)

 

We use ExifTool to update RAW files, as it's the mostly safe way to update RAW images (my respect to Phil Harvey). And it might be a reason of slow RAW image updating.

 

Could you please let me know:

- Whether you use shared or local catalogs?

- How many files did you import to your catalog?

- Your PC configuration (including your network speed)

 

For the Daminion Standalone version could you please launch your Daminion via command line with enabled logging:

- Press the Start button

- Type in the below line:

"c:\Program Files (x86)\Daminion Software\Daminion\Daminion.exe" -log_enable

- Press the Ctrl + F12 keys

- Try to update a few RAW images and then send me the "1.0\Logs" folder.

 

Or you can also give me a remote (controlled) access to your PC via TeamViewer so I can check it remotelly.

 

 

Tip: You can switch the auto-sync option off before large tag assignment operations, and the switch it On again. In this case Daminion will updates an image once for multiple tag assignments.

[Per-BKWine]

I did use the option not to move files.

 

"a lot of Cataloguing errors": this refers to the question in my other post. There are many images that have a question mark on the tumbnail and some that have a broken chain icon instead of a thumbnail. I am assuming that this indicates incomplete cataloguing. (Can't find any info on that in the Wiki manual so I am guessing)

 

This does not concern RAW files. (I think you should not embed in RAW files.) It is all tiff files, mainly of ~50 MB size. Accessed over a powerline network (max speed ~200 mb?), not super-fast gigabit nwk but still a pretty decent speed (see the info at the end on the comparison with another DAM).

 

Q: Whether you use shared or local catalogs?

A: A single local private catalogue. I'm just starting this so I use the simplest setup.

 

Q: How many files did you import to your catalog?

A: Around 3000.

 

But when I tested *embedding* to the files I just chose three files to embed to, not the whole catalogue!

 

Q: Your PC configuration (including your network speed):

A: Dell Precision M6700, network mentioned above, server is a Netgear ReadyNAS NV+.

 

But perhaps more importantly:

 

I did a comparative test, embed metadata to three files:

a) with Daminon

b) with Extensis Portfolio

with the exact same setup: local catalogue, files on the NAS.

 

Daminon: after ~30 minutes the progress was still on "processing file 1 of 3".

Portfolio: the three files were all updated in a few minutes time.

 

I can do the logging. (But it's not on RAW files, as I mentioned.)

[Murat]

Per, thanks a lot for the info!

 

Unfortunatelly I can't reproduce it with my test TIFF collection. May I ask you to send me a sample TIFF file and your catalog file for analizing?

[Murat]

Daminon: after ~30 minutes the progress was still on "processing file 1 of 3".

 

This is definetelly because of an error while metadata syncing. And after fixing this error the import speed will be much better: 130MB NAS based file was updated for 30 secs on my confguration.

[Per-BKWine]

Did some more testing of this comparing:

- network connection over power-line network (relatively fast, nominally 200 Mbps)

- direct ethernet over a cable and bridge (faster, nominally gigabit)

 

It did make a substantial difference:

 

--Cataloguing:

 

Cataloguing new files, a test with a folder on a NAS server containing ~100 tiff files each 50 MB

Powerline networking: ~30 minutes to catalogue the 100 files

Direct ethernet: 13 minutes in one test, 8 minutes in another.

 

The difference seems not unreasonable although it will take quite a long time if I am to catalogue all my images.

 

--Embedding:

 

Here the difference was more intriguing and the result was strange.

 

With a direct ethernet connection embedding in one file was almost instantaneous.

 

Embedding in two files took less than ten seconds.

 

If instead I connected over the powerline networking:

 

Embedding in one file: a little less than 30 seconds.

 

Embedding in two files (selecting two and then do 'write to file'): After waiting 30 minutes I cancelled. It was still showing the same message "processing file 1 of 2" (not sure of the exact wording.

 

There must be something strange going on here.

 

Follow-up:

 

I will later (probably overnight tonight) test cataloguing a larger number of files using the wired ethernet connection and see if I get the same cataloguing errors as I did previously (using the powerline network).

[Per-BKWine]

I catalogued a set of ~3000 images (tiffs, 50 MB mainly) being connected to the network and fileserver through a wired ethernet connection.

 

It went faster (launched it last night and was finished this morning, so less than 8 hours).

 

I did not get any "?" or broken-link icons, as far as I can tell. (But it is difficult to know 100% since I can't check every single image.)

 

So, yes, it went much better.

 

Perhaps Daminion struggles when the network connection is not super-fast (as with the powerline network) and then gets cataloguing errors, although the network is not actually down?

[Murat]

Perhaps Daminion struggles when the network connection is not super-fast (as with the powerline network) and then gets cataloguing errors, although the network is not actually down?

 

We'll try to reproduce this environment. But to be honest I never have a deal with the Powerline networks. What's an average file transfer speed over the powerline network between your NAS and your local computer?

[Per-BKWine]

Not sure how I could measure the transfer speed, but in principle the ethernet is a Gigabit ethernet (so has 1Gb speed theoretically) and the powerline connection has nominally 200 Mbit/s, which is much better that a normal wifi connection I think.

 

I did a test:

 

I transferred a 4.5 GB (giga byte) file (one single file) from my computer to the file server with the two types of connections. Here how long time it took:

 

Powerline connection: ~12 minutes

Ethernet cable: ~4 minutes

 

So, a clear difference but not orders of magnitude.

 

Perhaps that tells you something useful?

 

-Per

[Murat]

Powerline connection: ~12 minutes

Ethernet cable: ~4 minutes

So, a clear difference but not orders of magnitude.

Perhaps that tells you something useful?

 

Thanks for the tests!

 

I had some time to read a few articles on this weekend about Powerline connections - and I guess the speed shouldn't be a problem, but the stability of connection might be. Unfortunately we didn't have powerline adapters to reproduce the issue.

 

Some thoughts: we use the system "Ping" command to check the availability of files (to avoid freezing while checking remote unavaiable resources), and probably the Ping command works unstable on Powerline network configuration. But as I said before - I can't check it now