Any recommended settings to stop ehentai IP bans?

Ask questions and have them answered!
Post Reply
tewcsmcduke
Posts: 11
Joined: Fri Feb 05, 2021 7:16 pm

Any recommended settings to stop ehentai IP bans?

Post by tewcsmcduke » Tue Nov 16, 2021 9:47 pm

Pretty much the title. This program results in frequent IP bans which is annoying, but with a donator account (IP ban = account ban, no matter how many times you switch IP) it quickly makes the program borderline unusable. Any settings you recommend to avoid this? I play around a lot with download delay but I don't know which settings will stop the bans.

CyCaDeLiC
Posts: 3
Joined: Mon Dec 20, 2021 4:20 pm

Re: Any recommended settings to stop ehentai IP bans?

Post by CyCaDeLiC » Mon Dec 20, 2021 4:28 pm

Same question here. I also asked on the e-hentai forums, hoping that someone knows the internal settings for the "excessive pageviews" ban. I'll update my post here if I get a helpful reply.

My own experience so far:

From the e-hentai wiki: "Keep the number of subsequent threads to 4 or less."

At first I thought this would mean "4 or less simultaneous downloads", however that ran me into a temporary ban very quickly, even at 3 simultaneous page downloads (at a speed of one page for every 0.3 seconds, so 3x3 x 0.3s = ~9.3 pages per second).

I've currently restricted the speed to a single page download every 0.4 seconds, which seems to be running fine, but is arguably way slower than downloading the archived galleries manually. I can understand that these numbers might still appear "fast" for free users of e-hentai, but as a donator on the site it's easily possible to download enormous data volumes in a quick time (via their archive links). So, in theory, the traffic we generate via individual pageviews shouldn't be treated as "excessive" this quickly by their system.

Then again, I'm almost certain the issue is on their side and isn't caused by the HDoujin Downloader. Either way, I'll keep asking & testing around too.

User avatar
Squidy
Site Admin
Posts: 1079
Joined: Fri Mar 10, 2017 9:28 pm
Contact:

Re: Any recommended settings to stop ehentai IP bans?

Post by Squidy » Tue Dec 21, 2021 4:06 am

As discussed, setting a per-image download delay under Tools > Preferences > Timing & Delays is normally what I would recommend. Thank-you to CyCaDeLiC for providing some actual numbers.

HDD will only download one image at a time per task, so it's surprising that despite what it says in the wiki the bans are still occurring with only 3 simultaneous downloads. It seems that the actual restrictions are less lenient than the wiki suggests.
I'm the admin and developer of HDoujin Downloader.

CyCaDeLiC
Posts: 3
Joined: Mon Dec 20, 2021 4:20 pm

Re: Any recommended settings to stop ehentai IP bans?

Post by CyCaDeLiC » Sun Dec 26, 2021 12:33 pm

Update on my original post:

Personal tests:
I ran HDD without problems for four days in a row at a setting of 0.2 seconds delay between each page download from e(x)hentai. Single page download, not multiple at once. The download delay between gallery downloads was set to 12 seconds.

This worked fine up until today, when I received another IP ban, which leads me to believe that the restrictions might even depend on their total site traffic, potentially limiting users harder when the site is under a lot of stress (as could be the case during the holidays). I now upped the delay to 0.4 seconds again, but I almost expect I'll run into yet another ban eventually. The reason for this might be related to the following info I received on their forums:

E-hentai forums feedback:
A moderator informed me that using Download Managers, such as HDD, is strongly discouraged. For this reason I was told I'd also not receive an answer to my question regarding a recommended setting for HDD but should just stop using it. Apparently, the wiki entry is misleading and only refers to simultaneous archive downloads, not page downloads. This is worded poorly on the wiki, and there's also not a single statement about them discouraging the use of Download Managers, but I was told these "hammer" the servers and thus get punished by bans. They offered me alternative download scripts that access the archive and torrent files, but I must say I vastly prefer HDD over the script-method - even more so since the script options don't write the metadata into a separate file.

The funny, or maybe sad, thing about their restrictions: Every registered user has an image viewing limit on their website. In my case, that's a limit of 50.000 images I can view (or download) before the server would block me. This limit resets over time, and even at my aforementioned setting of 0.2 s/page and 12 s/gallery I actually can't reach the limit. At least not without letting HDD run 24 hours a day. This puts their enforced bans ad absurdum, or at least renders the image viewing limit an unreliable source of information (since I originally tried balancing the download speed with the image limit, to create a nice flow of never hitting the limit while still retaining a decent download rate - but that's sadly just not possible).

Conclusion:
I'll stick to HDD, and would ask if it's possible to maybe integrate such archive/torrent download functions into the software. Since HDD already connects to e(x)hentai with a user's account (login + cookies), it might be possible to send an archive/torrent download request to the server? In that case, users would be given three different methods of downloading, which could greatly mitigate the risk of running into an "excessive pageviews" ban. I can understand though if integrating such a feature just for one website would be too excessive / doesn't justify the workload. For me, e(x)hentai is the best platform to go to, I just wish they'd be less harsh with their bans.

Update 2:

And I received another temporary ban. This time upon running a script to grab metadata (since archive downloads on the website itself don't come with metadata files). The script was set to grab gallery data every four seconds (four seconds of "sleep" between sending a request to the server). HDD wasn't running at this time, and the script was the only active instance of accessing the website. Apparently, even such a long delay between requests is still regarded as "excessive". I'd now say that interacting with e(x)hentai in any form other than downloading archives/torrents from the website itself should only be done very scarcely, if at all. The bans just get longer each time, resulting in a big amount of lost time. Even just downloading images manually would be quicker at this point - and none of this is the fault of HDD. As was proven by getting banned by running an unrelated script, it's the website itself that has extremely strict limitations set in place. I don't think there's anything Squidy can do to work around this, other than the aforementioned archive/torrent file request.

tewcsmcduke
Posts: 11
Joined: Fri Feb 05, 2021 7:16 pm

Re: Any recommended settings to stop ehentai IP bans?

Post by tewcsmcduke » Wed Dec 29, 2021 7:11 pm

Before my e-hentai downloads broke completely for whatever unknown reason, I found that I was able to eliminate bans by reducing the number of times to retry failed pages to 1, changing delay between pages to 1.2, between galleries to 1.5 and between downloads to 10s, I was able to more or less eliminate bans. These numbers aren't "fast" by any means but they do seem to have stopped bans before HDD decided to just not work pretty much at all on this website.

CyCaDeLiC
Posts: 3
Joined: Mon Dec 20, 2021 4:20 pm

Re: Any recommended settings to stop ehentai IP bans?

Post by CyCaDeLiC » Fri Dec 31, 2021 1:06 pm

tewcsmcduke wrote:
Wed Dec 29, 2021 7:11 pm
Before my e-hentai downloads broke completely for whatever unknown reason, I found that I was able to eliminate bans by reducing the number of times to retry failed pages to 1, changing delay between pages to 1.2, between galleries to 1.5 and between downloads to 10s, I was able to more or less eliminate bans. These numbers aren't "fast" by any means but they do seem to have stopped bans before HDD decided to just not work pretty much at all on this website.
I've been running HDD on e-h since my last post now without issues, on the following settings:
- 1 simultaneous download
- 0.5 seconds delay between pages
- 11 seconds delay between downloads
- Times to retry failed pages: 0

Site specific:
- 4 seconds delay between adding e-h galleries from the clipboard to the queue
- Image size: Original
- Download from mirror sites: Yes

Might give this a try and see if it works - I'm happy with it so far.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest