Jump to content

Performance over the network.


SergeS
 Share

Recommended Posts

Seems like Daminion server has very low performance when working over the network, why it could be?

 

My configuration:

Daminion server installed on my desktop PC (i7-4790k, 16Gb ram, win8.1-64bit), photos located on the Synology NAS, both have wired gigabit network. Daminion server has a catalog with around 75K photos. I am tagging one folder (all photos are NEF from Nikon D810, totally 1855 files take 70GBytes, each file is around 40MB). Tagging (syncing) takes around 8 hours, approx 1 photo in 15 seconds!

 

I thought the bottleneck is network, but network performance is far from saturation, see screenshot

 

I have tried to copy all this folder to local harddrive (takes around 20 minutes, 60MB/Sec), add them to local catalog (10 minutes), tag them locally (1 hour) and copy them back to the NAS (again around 20 minutes), so totally less then two hours. It is four times faster.

 

What could be done to significantly improve performance over the network?

Why network in very under-load during tagging over the network? Where is the bottleneck in this case?

 

And main question: is this possible to have postgresql server not on the PC with Daminion-server, but on the NAS, where postgresl server is already installed (all Synology NAS'es have it installed) and where all pictures are located? Will it increase performance (over the network) by avoiding transfer all big photo-files back and forth?

Link to comment
Share on other sites

Using a PostgreSQL from another computer/device is possible (but a bit tricky), but this will not impact on metadata syncing performance.

 

- What options are checked in your Metadata Writing Preference of your Daminion Server administration panel? Can you show me a screenshot?

 

- Do you see the same performance with syncing metadata from locally copied NEF files?

 

- Can you please send me sample NEF file to support/at/daminion/dot/net

Link to comment
Share on other sites

Using a PostgreSQL from another computer/device is possible (but a bit tricky), but this will not impact on metadata syncing performance.

Why? I thought the bottleneck is copying files over the network, having database server on the same computer as main storage should help to eliminate this issue, didn't I? Where is the bottleneck, why so slow?

 

- What options are checked in your Metadata Writing Preference of your Daminion Server administration panel? Can you show me a screenshot?

Yes, I have the slowest possible options selected there (screenshot) :-), but this is what I need, I would like to keep all my tags to be embedded into photos.

Anyway, changing these options may help to speed up (paid by loosing some functionality), but will not explain why so slow with this selection.

 

- Do you see the same performance with syncing metadata from locally copied NEF files?

I've describing it in my first email, syncing over the network is approx. 8 times slowly (8 hours vs 1 hour).

Full-cycle-time (include copying over the network to local PC, creating new catalog with these photos, syncing and copying them back) takes around 2 hours which is still 4 times less than doing the same over the network :-).

 

- Can you please send me sample NEF file to support/at/daminion/dot/net

I've put some samples here. Will also send sample to support/at/daminion/dot/net

 

Thank you.

Link to comment
Share on other sites

Hello SergeS,

 

the performance of Daminon actions using a network connection for files on a NAS is an issue for me since 2012. There were some improvments since 2012 made by the Daminion team.

I made the same tests in the past (workflow by Daminion or copy to local drive...) and got similar results. All my files are on a NAS and the transfer performance of the Windows Copy function by Total Commander is max.114MB/sec for videos (2-4GB) and 95MB/sec for DNG (25MB) from NAS to local SSD and back.

The topic performance is on the roadmap of the upcoming release 5.0.

I'll try to repeat my previous tests with the current build.

I would like to create test cases (100files/test case if possible) like:

01 Images

0101 JPG

0102 TIF

010201 TIF

010202 TIF >100MB

0103 Vector Images

0104 PSD/PSB

0105 RAW

010501 CR2, NEF...

010502 DNG

01050201 DNG extern created by Adobe DNG converter

01050202 DNG created by the Daminion Import

02 Videos

03 Audio

0301 FLAC

0302 MP3

04 Documents

05 other media formats

 

Regards, Uwe

Link to comment
Share on other sites

What kind of Synology NAS are we talking about?

 

I have a Synology DS211j. I have had it for several years and it is extremely slow and underpowered. Depending on your model, what you are seeing may be appropriate.

 

I have a copy of my files on a regular hard drive that Daminion works on. I then back these files up to the Synology NAS and another device.

 

Frank Hahn

Link to comment
Share on other sites

What kind of Synology NAS are we talking about?

 

I have a Synology DS211j. I have had it for several years and it is extremely slow and underpowered. Depending on your model, what you are seeing may be appropriate.

 

I have a copy of my files on a regular hard drive that Daminion works on. I then back these files up to the Synology NAS and another device.

 

Frank Hahn

 

My main home nas is Xpenology (Synology DSM software, but HP Microserver Gen7 hardware, with upgraded memory and network card), and I have no performance problem with other applications as far as with transfer files over the network, it gives full wired gigabit speed with no problem.

 

Also I have Synology DS410 at home, the same situation, there are no problem with other apps, but very slow with Daminion, and network broadband seems to be underused.

 

And, finally, I have few Synology nas'es at work (DS211J and DS214+), I did not try with them, may try if needed.

Link to comment
Share on other sites

My Server is a HP DataVault (StorageWorks) X310 mit 4GB RAM 6TB of Disc Space and a Intel Atom Dual Core (1.6GHz).

 

This was actually ment as the backup server but since the main server just a tower with windows 7 was so time consuming to manage i used that one. just need a faster now.

 

it does it's job, however sometimes I wish to have more power.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...