Loading, importing and editing times

Fixed bugs will be filed here.
DuLac
Posts: 213
Joined: Sun Feb 10, 2013 9:53 am

Re: Loading, importing and editing times

Post by DuLac »

The log files is at the following link https://copy.com/CDEheBQdeUhT.
musoware
Site Admin
Posts: 1847
Joined: Fri Sep 14, 2012 6:50 am

Re: Loading, importing and editing times

Post by musoware »

So we're back to where we started. You were getting improved performance because it wasn't your full library?
I'll put some more logging in to see if we can find out where the bottle-necks are.
musoware
Site Admin
Posts: 1847
Joined: Fri Sep 14, 2012 6:50 am

Re: Loading, importing and editing times

Post by musoware »

OK please download and install 2.0.02 and start it up in debug mode: Muso 2 now writes to muso2.log, not muso.log. There's some extra logging in the startup which outputs a table which details where the time is going. Mine all works in 8 seconds or so, this includes reading the library xml file, the reference xml, then parsing the track collection and collating close to 50,000 tracks into 5,255 albums, plus building other data collections :

Code: Select all

+---------+-------+------+-------------------------------------------------
| TotSecs | Count |Range*| Detail
+---------+-------+------+-------------------------------------------------
|    .000 |     1 |.     | InitDb folder={0}
|    .000 |     1 |.     | InitDb start
|    .006 |     1 |.     | InitDb 1
|    .012 |     1 | .    | InitDb 2
|    .000 |     1 |.     | InitDb 3
|    .000 |     1 |.     | Loading shared database C:\Users\Public\Music\muso\database\Music Library.xml
|   3.389 |     1 |     .| done read Music db
|         |     1 |.     | read Reference db {0}
|   0.200 |     1 |   .  | done read Reference db
|   0.071 |     1 |  .   | LoadClassicalData
|    .017 |     1 | .    | SavereferenceDB : no changes
|    .000 |     1 |.     | Collate from initDB
|    .008 |     1 |.     | CollateAlbums start
|    .000 |     1 |.     | Parse {0} tracks
|   0.048 | 49859 |*     | . Track {0} instantiate
|   0.730 | 49859 |*+    | . albumKey {0}
|    .004 |  5255 |*     | . 1st track of new album
|   0.041 | 49859 |*     | . add to _tracks
|   0.036 | 49859 |*     | . add size {0}
|   0.031 | 44604 |*     | . add track to existing album
|         |     1 |.     | Populate collections from {0} albums
|   0.957 |  5255 |*.    | . {0}/{1}
|   1.088 | 49859 |*     | . . set track artist links to album
|   0.098 |  5255 |*     | . get total time
|    .014 |  5255 |*     | . total time {0}
|    .007 |  5255 |*     | . check album qualification tracks=7 time=40
|    .005 |  5255 |*     | . add to _allAlbums
|   0.221 |  5255 |*     | . deduce album category
|    .007 |  1376 |*     | . album category = NonAlbum
|   0.105 |  5255 |*     | . add to artist groups
|   0.030 |  5255 |*     | . register other contributing artists
|   0.228 |  5255 |*.    | . genre groups
|   0.036 |  5255 |*     | . composer groups
|   0.338 |  5255 |*     | . conductor groups
|    .010 |  5255 |*     | . band groups
|    .008 |  5255 |*     | . performer groups
|    .008 |  5255 |*     | . month groups
|   0.027 |  5255 |*     | . year groups
|    .017 |  2649 |*     | . album category = FullAlbum
|    .004 |   882 |*     | . album category = NonQualifyingAlbum
|    .002 |   348 |*     | . album category = Compilation
|         |     1 |.     | ResumeUpdateID3
|         |     1 |.     | CollateAlbums end
|         |     1 |.     | done collate
|   0.460 |     1 |   .  | InitDb end
+---------+-------+------+-------------------------------------------------
    8.297 (00:00:08.2970096)

*Range pips are [1]<.01s [2]<.05s [3]<.1s [4]<.5s [5]<1s [6]>=1s, ('.'=once, '+'<10, '*'=10+)
It's going to be interesting to analyse why yours takes over 20 minutes for this process for a similar sized database and 4x the memory!
DuLac
Posts: 213
Joined: Sun Feb 10, 2013 9:53 am

Re: Loading, importing and editing times

Post by DuLac »

musoware wrote:So we're back to where we started. You were getting improved performance because it wasn't your full library?
I'll put some more logging in to see if we can find out where the bottle-necks are.
You are right. Looking back I remember that I tried importing some files to a fresh db and that made all the difference. My fault, sorry!
I am now about to try muso 2.0.2 as suggested and I'll post the log file as soon as available
In the meantime you will find at the following link a new log file, still from ver. 2.0.0. It will show how I normally use muso. I started the program, imported some tracks, played some music and edited a couple of tags. All these actions except playing music files take an extremely long time to execute. It is a much bigger file as I left muso running overnight.
https://copy.com/JNasDcMjK82H
musoware
Site Admin
Posts: 1847
Joined: Fri Sep 14, 2012 6:50 am

Re: Loading, importing and editing times

Post by musoware »

Well this is the slightly worrying thing. If there's no single reason for the bottle-neck and your system runs generally slowly there's maybe not much that can be done, other than to say that this is abnormal in the extreme for a machine of your spec!

Do other programs run slowly?
Does muso start up and run any faster after a clean reboot with no other foreground apps running?
Are there any large memory or cpu hungry background apps running?
Does the resource monitor show anything interesting as muso starts? Mine shows Muso.exe using around 144Mb private bytes on the memory tab after it has fully started.
Is there any memory throttling software on your machine that limits memory per application which might cause lots of swapping to disk?
RichG
Posts: 215
Joined: Fri Feb 01, 2013 4:46 pm

Re: Loading, importing and editing times

Post by RichG »

Could Anti-Virus software be getting in the way?
DuLac
Posts: 213
Joined: Sun Feb 10, 2013 9:53 am

Re: Loading, importing and editing times

Post by DuLac »

musoware wrote:OK please download and install 2.0.02 and start it up in debug mode: Muso 2 now writes to muso2.log, not muso.log. There's some extra logging in the startup which outputs a table which details where the time is going.
Please collect the new log file from https://copy.com/cNseCLlYYAU3. I wish you luck! Many thanks ;-)
DuLac
Posts: 213
Joined: Sun Feb 10, 2013 9:53 am

Re: Loading, importing and editing times

Post by DuLac »

RichG wrote:Could Anti-Virus software be getting in the way?
Re: Antivirus. I already thought about it. I use Microsoft Security Essentials together with Malwarebytes Anti-Malware. I am going to try loading the application with permutations of these two disconnected.
DuLac
Posts: 213
Joined: Sun Feb 10, 2013 9:53 am

Re: Loading, importing and editing times

Post by DuLac »

musoware wrote:Well this is the slightly worrying thing. If there's no single reason for the bottle-neck and your system runs generally slowly there's maybe not much that can be done, other than to say that this is abnormal in the extreme for a machine of your spec!

Do other programs run slowly?
Does muso start up and run any faster after a clean reboot with no other foreground apps running?
Are there any large memory or cpu hungry background apps running?
Does the resource monitor show anything interesting as muso starts? Mine shows Muso.exe using around 144Mb private bytes on the memory tab after it has fully started.
Is there any memory throttling software on your machine that limits memory per application which might cause lots of swapping to disk?
No, other programs run fine. I use Photoshop a lot and I work with very very large files
A clean reboot does not seem to make any difference as far as muso is concerned
I run Win7 with all graphical gimmicks turned off (layout similar to ol' Win95) to avoid using system resources.
Yesterday I monitored network using Windows Task Manager. When I start muso there is a peek in the bytes sent and bytes received. After a little while the traffic slows to almost none. I guess that is what the log files show too.
Memory throttling software, none that I can think of.
In the meantime I tried to start muso with no antiviruses programs turned on. Nothing changed. I am still waiting for muso to start.

I really appreciate your help and suggestions. I am going to mentally prepare to re-install Windows and start afresh. Cheers
musoware
Site Admin
Posts: 1847
Joined: Fri Sep 14, 2012 6:50 am

Re: Loading, importing and editing times

Post by musoware »

DuLac wrote:
musoware wrote:OK please download and install 2.0.02 and start it up in debug mode: Muso 2 now writes to muso2.log, not muso.log. There's some extra logging in the startup which outputs a table which details where the time is going.
Please collect the new log file from https://copy.com/cNseCLlYYAU3. I wish you luck! Many thanks ;-)
Well comparing with mine there's 2 main code section bottle-necks there, plus a minor 3rd. That's much better than all steps being slower, so gives me something to look into. Those steps aren't consistently slower though, as the range of timings shows. I may have to add some more debug in those 3 areas to narrow it down further.
Post Reply