Page 3 of 4

Re: Loading, importing and editing times

Posted: Fri Feb 28, 2014 5:23 pm
by DuLac
The log files is at the following link https://copy.com/CDEheBQdeUhT.

Re: Loading, importing and editing times

Posted: Fri Feb 28, 2014 11:55 pm
by musoware
So we're back to where we started. You were getting improved performance because it wasn't your full library?
I'll put some more logging in to see if we can find out where the bottle-necks are.

Re: Loading, importing and editing times

Posted: Sat Mar 01, 2014 12:52 am
by musoware
OK please download and install 2.0.02 and start it up in debug mode: Muso 2 now writes to muso2.log, not muso.log. There's some extra logging in the startup which outputs a table which details where the time is going. Mine all works in 8 seconds or so, this includes reading the library xml file, the reference xml, then parsing the track collection and collating close to 50,000 tracks into 5,255 albums, plus building other data collections :

Code: Select all

+---------+-------+------+-------------------------------------------------
| TotSecs | Count |Range*| Detail
+---------+-------+------+-------------------------------------------------
|    .000 |     1 |.     | InitDb folder={0}
|    .000 |     1 |.     | InitDb start
|    .006 |     1 |.     | InitDb 1
|    .012 |     1 | .    | InitDb 2
|    .000 |     1 |.     | InitDb 3
|    .000 |     1 |.     | Loading shared database C:\Users\Public\Music\muso\database\Music Library.xml
|   3.389 |     1 |     .| done read Music db
|         |     1 |.     | read Reference db {0}
|   0.200 |     1 |   .  | done read Reference db
|   0.071 |     1 |  .   | LoadClassicalData
|    .017 |     1 | .    | SavereferenceDB : no changes
|    .000 |     1 |.     | Collate from initDB
|    .008 |     1 |.     | CollateAlbums start
|    .000 |     1 |.     | Parse {0} tracks
|   0.048 | 49859 |*     | . Track {0} instantiate
|   0.730 | 49859 |*+    | . albumKey {0}
|    .004 |  5255 |*     | . 1st track of new album
|   0.041 | 49859 |*     | . add to _tracks
|   0.036 | 49859 |*     | . add size {0}
|   0.031 | 44604 |*     | . add track to existing album
|         |     1 |.     | Populate collections from {0} albums
|   0.957 |  5255 |*.    | . {0}/{1}
|   1.088 | 49859 |*     | . . set track artist links to album
|   0.098 |  5255 |*     | . get total time
|    .014 |  5255 |*     | . total time {0}
|    .007 |  5255 |*     | . check album qualification tracks=7 time=40
|    .005 |  5255 |*     | . add to _allAlbums
|   0.221 |  5255 |*     | . deduce album category
|    .007 |  1376 |*     | . album category = NonAlbum
|   0.105 |  5255 |*     | . add to artist groups
|   0.030 |  5255 |*     | . register other contributing artists
|   0.228 |  5255 |*.    | . genre groups
|   0.036 |  5255 |*     | . composer groups
|   0.338 |  5255 |*     | . conductor groups
|    .010 |  5255 |*     | . band groups
|    .008 |  5255 |*     | . performer groups
|    .008 |  5255 |*     | . month groups
|   0.027 |  5255 |*     | . year groups
|    .017 |  2649 |*     | . album category = FullAlbum
|    .004 |   882 |*     | . album category = NonQualifyingAlbum
|    .002 |   348 |*     | . album category = Compilation
|         |     1 |.     | ResumeUpdateID3
|         |     1 |.     | CollateAlbums end
|         |     1 |.     | done collate
|   0.460 |     1 |   .  | InitDb end
+---------+-------+------+-------------------------------------------------
    8.297 (00:00:08.2970096)

*Range pips are [1]<.01s [2]<.05s [3]<.1s [4]<.5s [5]<1s [6]>=1s, ('.'=once, '+'<10, '*'=10+)
It's going to be interesting to analyse why yours takes over 20 minutes for this process for a similar sized database and 4x the memory!

Re: Loading, importing and editing times

Posted: Sat Mar 01, 2014 9:16 am
by DuLac
musoware wrote:So we're back to where we started. You were getting improved performance because it wasn't your full library?
I'll put some more logging in to see if we can find out where the bottle-necks are.
You are right. Looking back I remember that I tried importing some files to a fresh db and that made all the difference. My fault, sorry!
I am now about to try muso 2.0.2 as suggested and I'll post the log file as soon as available
In the meantime you will find at the following link a new log file, still from ver. 2.0.0. It will show how I normally use muso. I started the program, imported some tracks, played some music and edited a couple of tags. All these actions except playing music files take an extremely long time to execute. It is a much bigger file as I left muso running overnight.
https://copy.com/JNasDcMjK82H

Re: Loading, importing and editing times

Posted: Sat Mar 01, 2014 9:58 am
by musoware
Well this is the slightly worrying thing. If there's no single reason for the bottle-neck and your system runs generally slowly there's maybe not much that can be done, other than to say that this is abnormal in the extreme for a machine of your spec!

Do other programs run slowly?
Does muso start up and run any faster after a clean reboot with no other foreground apps running?
Are there any large memory or cpu hungry background apps running?
Does the resource monitor show anything interesting as muso starts? Mine shows Muso.exe using around 144Mb private bytes on the memory tab after it has fully started.
Is there any memory throttling software on your machine that limits memory per application which might cause lots of swapping to disk?

Re: Loading, importing and editing times

Posted: Sat Mar 01, 2014 10:02 am
by RichG
Could Anti-Virus software be getting in the way?

Re: Loading, importing and editing times

Posted: Sat Mar 01, 2014 10:19 am
by DuLac
musoware wrote:OK please download and install 2.0.02 and start it up in debug mode: Muso 2 now writes to muso2.log, not muso.log. There's some extra logging in the startup which outputs a table which details where the time is going.
Please collect the new log file from https://copy.com/cNseCLlYYAU3. I wish you luck! Many thanks ;-)

Re: Loading, importing and editing times

Posted: Sat Mar 01, 2014 10:22 am
by DuLac
RichG wrote:Could Anti-Virus software be getting in the way?
Re: Antivirus. I already thought about it. I use Microsoft Security Essentials together with Malwarebytes Anti-Malware. I am going to try loading the application with permutations of these two disconnected.

Re: Loading, importing and editing times

Posted: Sat Mar 01, 2014 10:46 am
by DuLac
musoware wrote:Well this is the slightly worrying thing. If there's no single reason for the bottle-neck and your system runs generally slowly there's maybe not much that can be done, other than to say that this is abnormal in the extreme for a machine of your spec!

Do other programs run slowly?
Does muso start up and run any faster after a clean reboot with no other foreground apps running?
Are there any large memory or cpu hungry background apps running?
Does the resource monitor show anything interesting as muso starts? Mine shows Muso.exe using around 144Mb private bytes on the memory tab after it has fully started.
Is there any memory throttling software on your machine that limits memory per application which might cause lots of swapping to disk?
No, other programs run fine. I use Photoshop a lot and I work with very very large files
A clean reboot does not seem to make any difference as far as muso is concerned
I run Win7 with all graphical gimmicks turned off (layout similar to ol' Win95) to avoid using system resources.
Yesterday I monitored network using Windows Task Manager. When I start muso there is a peek in the bytes sent and bytes received. After a little while the traffic slows to almost none. I guess that is what the log files show too.
Memory throttling software, none that I can think of.
In the meantime I tried to start muso with no antiviruses programs turned on. Nothing changed. I am still waiting for muso to start.

I really appreciate your help and suggestions. I am going to mentally prepare to re-install Windows and start afresh. Cheers

Re: Loading, importing and editing times

Posted: Sat Mar 01, 2014 12:52 pm
by musoware
DuLac wrote:
musoware wrote:OK please download and install 2.0.02 and start it up in debug mode: Muso 2 now writes to muso2.log, not muso.log. There's some extra logging in the startup which outputs a table which details where the time is going.
Please collect the new log file from https://copy.com/cNseCLlYYAU3. I wish you luck! Many thanks ;-)
Well comparing with mine there's 2 main code section bottle-necks there, plus a minor 3rd. That's much better than all steps being slower, so gives me something to look into. Those steps aren't consistently slower though, as the range of timings shows. I may have to add some more debug in those 3 areas to narrow it down further.