queue.manga loading very slow.

Run into a problem? Report it here!
Post Reply
wawawa

queue.manga loading very slow.

Post by wawawa » Sun Aug 13, 2017 1:13 pm

Hello.

I update Version: 1.16.9.32 (Release 254).
Loading of "queue.manga" slowed down from this version.
I waited for 4 minutes.
My "queue.manga" size is 399166 bytes.
Since "queue.manga" is too big, did the loading slow down?
It seems that there was no problem with the old version.

User avatar
Squidy
Site Admin
Posts: 1270
Joined: Fri Mar 10, 2017 9:28 pm
Contact:

Re: queue.manga loading very slow.

Post by Squidy » Tue Aug 15, 2017 7:38 am

399166 bytes should correspond to approximately ~900 queue items. That's not a lot, relatively speaking, and HDoujin Downloader should be able to handle that many items without an issue. Usually slowdown starts to occur around ~10 000 items.

Do you remember what release you were using before updating? I didn't make any changes to how the queue is handled between Release 253 and 254, so I don't think it's the update that caused it (assuming you were using Release 253 before the issue started).

For a number of reasons, it's possible for HDoujin to take a moment to start up. For example, this can happen if HDoujin has trouble accessing the internet-- It checks for updates on start-up by default, and that can take a moment if your internet is running slow. I think that this is probably what was happening, and if so, it'll fix itself when your internet connection is working up-to-speed again.

If it hasn't resolved itself and you've been experiencing this issue consistently since it started, let me know, and we can look into it further.

One thing you can try in order to verify the cause of the issue is moving your queue file to a different directory (queue.manga), and then starting HDoujin without it. You can then import the queue again using "File > Open". If importing the queue takes a long time, then we know it's not a start-up issue.
I'm the admin and developer of HDoujin Downloader.

wawawa

Re: queue.manga loading very slow.

Post by wawawa » Tue Aug 15, 2017 1:50 pm

>399166 bytes should correspond to approximately ~900 queue items. That's not a lot, relatively speaking, and HDoujin Downloader should be able to handle that many items without an issue. Usually slowdown starts to occur around ~10 000 items.

I understand.

>Do you remember what release you were using before updating? I didn't make any changes to how the queue is handled between Release 253 and 254, so I don't think it's the update that caused it (assuming you were using Release 253 before the issue started).

I have started the HDoujin Downloader twice a week.
However, I do not know if it was 253.

>One thing you can try in order to verify the cause of the issue is moving your queue file to a different directory (queue.manga), and then starting HDoujin without it. You can then import the queue again using "File > Open". If importing the queue takes a long time, then we know it's not a start-up issue.

My PC(win7 x64),Another PC(win10 x64),
I tried "File > Open" in both PC environments.
Both became "wait 4 minutes".

By the way, delete "queue.manga", HDOUJIN DOWNLOADER will start up quickly.
If other user are not experiencing the phenomenon, is my environment problem..?

thank you.

wawawa

Re: queue.manga loading very slow.

Post by wawawa » Tue Aug 15, 2017 3:41 pm

sorry. I mistook the size of "queue.manga". Correctly 3991666.

User avatar
Squidy
Site Admin
Posts: 1270
Joined: Fri Mar 10, 2017 9:28 pm
Contact:

Re: queue.manga loading very slow.

Post by Squidy » Tue Aug 15, 2017 4:03 pm

wawawa wrote:
Tue Aug 15, 2017 3:41 pm
sorry. I mistook the size of "queue.manga". Correctly 3991666.
In that case, how many items do you have in your queue? 3991666 corresponds to approximately ~11 000 items, right? That might be the source of the slowdown. Depending on how powerful your computer it is, it could take some time to load up that many items.

If you want, you can send me your "queue.manga" file and I'll import it myself and see if I can reproduce the problem. If so, I'll see where it's spending the most time and do what I can to reduce it. You can email it to me (to the email address displayed on the "About" tab of the program), or upload it to a file host and send it via the contact form. Either way, I'll take some time to do some stress testing at some point and find out what can be done.

For the record, if you leave completed items in your queue to keep track of what you've already downloaded, I recommend enabling history instead. You can use "Ctrl+H" to open the history panel (or go to "Downloads > History"), and if you enable it, it will keep track of all completed downloads. If you click the button on the top right of the panel to access your history settings, there's also an option to avoid adding any URL to the queue that exists in history, making it easy to avoid accidentally downloading something you've downloaded before.
I'm the admin and developer of HDoujin Downloader.

wawawa

Re: queue.manga loading very slow.

Post by wawawa » Tue Aug 15, 2017 5:05 pm

It seems that leaving the queue was not a proper usage.

I decided to use the history function effectively.
It became very comfortable.

thank you.

User avatar
Riztard
Posts: 193
Joined: Mon Apr 10, 2017 12:48 pm
Location: Indonesia
Contact:

Re: queue.manga loading very slow.

Post by Riztard » Thu Aug 17, 2017 4:57 am

this give me some idea. thx
:hdoujin: THX for your Work Squidy. You are great. :hdoujin: by - Player 1

4TdE3VhSvz
Posts: 4
Joined: Sat Feb 09, 2019 12:54 pm

Re: queue.manga loading very slow.

Post by 4TdE3VhSvz » Sat Feb 09, 2019 1:26 pm

I've added around 9000 entries and it nearly kills the entire programm. It's inconvenient if I can easly add hundreds of thousands of links through the list generation tool but more than 1000 will pretty much unable to download. I would highly recommend to update the way the manga list works so it can support several ten thousands of mangas without performance problems. Some ways to acquire this would be:
  • Place the manga list into a separate frame that runs asynchronous from the mainframe and in a different thread.
  • Use special, prestored display objects, that will be used to render only the displayed things and nothing else. Everytime you start a specific entrys download or click to view more information you can load the full object and discard it afterwards
  • Preload only visible content of the list, if someone scrolls discard the content that has left the users view.
  • Use hashmaps(in c# the Dictionary is equivalent) instead of lists or even better an indexed sql database for the manga queue objects.
  • Use as much primitive types as possible (Dictionary<int, string[]> would be sufficient for the whole manga list)
I should also mention that I'd like to generate ~300000 links and download them afterwards. In this case even the generator nearly kills itself and therefore some of the above mentioned improvements could also be applied to the generator.

LieXXX
Posts: 24
Joined: Tue Oct 10, 2017 5:59 am

Re: queue.manga loading very slow.

Post by LieXXX » Sat May 25, 2019 3:59 am

I am experiencing a similar issue but for me it is when the program first opens with a lot of items in the queue.

LieXXX
Posts: 24
Joined: Tue Oct 10, 2017 5:59 am

Re: queue.manga loading very slow.

Post by LieXXX » Sun May 26, 2019 2:35 am

I found out that disabling "Mark duplicate items" under options solves the issue. The program opens a lot quicker.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests