8muses - download subdirectories

Ask questions and have them answered!
Post Reply
dartaron
Posts: 5
Joined: Thu Feb 10, 2022 1:07 pm

8muses - download subdirectories

Post by dartaron » Thu Feb 10, 2022 1:36 pm

HI,
there is a similar question in the forum but it is not answered and maybe it is not clearly phrased. So, here we go:

Let's say I want to download all Giantess-Club-Comics from https://comics.8muses.com/comics/album/ ... lub-Comics
The directory for the first folder goes as follows:

Code: Select all

https://comics.8muses.com/comics/album/Giantess-Club-Comics
https://comics.8muses.com/comics/album/Giantess-Club-Comics/Mrs_-Turner
https://comics.8muses.com/comics/album/Giantess-Club-Comics/Mrs_-Turner/Issue-1
https://comics.8muses.com/comics/album/Giantess-Club-Comics/Mrs_-Turner/Issue-2
.
.
https://comics.8muses.com/comics/album/Giantess-Club-Comics/Mrs_-Turner/Issue-8
Each issue contains several pages. When I download I get this directory structure:
Giantess Club Comics\001 - Mrs. Turner
This directory contains 8 files:

Code: Select all

Giantess Club Comics_0.jpg
.
.
.Giantess Club Comics_7.jpg
So instead of downloading each issue, it downloads only 1 page (probably because it does not go deeper).

How can I fix that? or is it a bug?

User avatar
Squidy
Site Admin
Posts: 1079
Joined: Fri Mar 10, 2017 9:28 pm
Contact:

Re: 8muses - download subdirectories

Post by Squidy » Fri Feb 11, 2022 10:06 am

As you noted, when you add a URL that contains several albums, the albums are added as chapters. They are not crawled recursively, so only the images immediately accessible from the album page are downloaded. It's not a bug, but perhaps an opportunity for a quality-of-life fix.

I just uploaded a module update that adds a new setting under Tools > Site preferences > 8muses called "Download albums recursively". When checked, albums will be crawled recursively and the images will be saved under subdirectories according to the directory structure they came from...

... At least, this'll happen once I put out a new program update. You can enable the setting before that, but it'll put images from all nested galleries into the same folder. There's a bug in the current release that makes it ignore image subdirectories-- It's such a rare case that I didn't notice it until now.
I'm the admin and developer of HDoujin Downloader.

dartaron
Posts: 5
Joined: Thu Feb 10, 2022 1:07 pm

Re: 8muses - download subdirectories

Post by dartaron » Fri Feb 11, 2022 6:55 pm

Thanks a lot for your so prompt response. I will be buying a subscription just because I appreciate such a prompt response from a developer.

Just a question. Is the bug you mentioned only in the current version? Meaning, if I use a previous version will it be able to not ignore image subdirectories?

User avatar
Squidy
Site Admin
Posts: 1079
Joined: Fri Mar 10, 2017 9:28 pm
Contact:

Re: 8muses - download subdirectories

Post by Squidy » Fri Feb 11, 2022 9:13 pm

Thanks for your support! ❤️
dartaron wrote:
Fri Feb 11, 2022 6:55 pm
Just a question. Is the bug you mentioned only in the current version? Meaning, if I use a previous version will it be able to not ignore image subdirectories?
The bug is present for all modules implemented in Lua regardless of version. Like I said, the need for this kind of recursion doesn't come up often, so this is the first time I've implemented it in a Lua module. No worries though, I just uploaded a new prerelease version with a fix you can download here: https://github.com/HDoujinDownloader/HD ... .1-win.zip

Just extract the archive over top of your current HDoujin Downloader directory.

Let me know if you have any issues with it! There is one bug I'm aware of where, if you have per-chapter conversion enabled, it will only convert the last subdirectory. You can read more about that here. If this is something you'd like me to prioritize, let me know.
I'm the admin and developer of HDoujin Downloader.

dartaron
Posts: 5
Joined: Thu Feb 10, 2022 1:07 pm

Re: 8muses - download subdirectories

Post by dartaron » Wed Feb 16, 2022 2:45 pm

Hi again, I tried it and it seems to work.Thanks a lot. One edgecase where not everything was downloaded was when subfolder_1 contained subfolder_2 and some seperate images. In this case only subfolder_2 is downloaded and the images are not....

Also, is there a selection where the subfolder structure is preserved instead of getting flattened?

User avatar
Squidy
Site Admin
Posts: 1079
Joined: Fri Mar 10, 2017 9:28 pm
Contact:

Re: 8muses - download subdirectories

Post by Squidy » Wed Feb 16, 2022 9:33 pm

dartaron wrote:
Wed Feb 16, 2022 2:45 pm
One edgecase where not everything was downloaded was when subfolder_1 contained subfolder_2 and some seperate images. In this case only subfolder_2 is downloaded and the images are not....
Could you send me a link to the gallery so I can check it out? I had considered this possibility, but wasn't able to find any applicable galleries.
dartaron wrote:
Wed Feb 16, 2022 2:45 pm
Also, is there a selection where the subfolder structure is preserved instead of getting flattened?
Did you download the prerelease version I linked in the above post? It will be flattened in earlier releases because the image subdirectory isn't respected, however it is not flattened if you use the latest prerelease:

Image
I'm the admin and developer of HDoujin Downloader.

dartaron
Posts: 5
Joined: Thu Feb 10, 2022 1:07 pm

Re: 8muses - download subdirectories

Post by dartaron » Thu Feb 17, 2022 1:21 pm

Squidy wrote:
Wed Feb 16, 2022 9:33 pm
dartaron wrote:
Wed Feb 16, 2022 2:45 pm
One edgecase where not everything was downloaded was when subfolder_1 contained subfolder_2 and some seperate images. In this case only subfolder_2 is downloaded and the images are not....
Could you send me a link to the gallery so I can check it out? I had considered this possibility, but wasn't able to find any applicable galleries.
in this gallery:
https://comics.8muses.com/comics/album/ ... com-Comics
this subdir:
https://comics.8muses.com/comics/album/ ... ing-in-Tim
contains both images & the subfolder
https://comics.8muses.com/comics/album/ ... Tim/Pinups

the images are not downloaded



Squidy wrote:
Wed Feb 16, 2022 9:33 pm
dartaron wrote:
Wed Feb 16, 2022 2:45 pm
Also, is there a selection where the subfolder structure is preserved instead of getting flattened?
Did you download the prerelease version I linked in the above post? It will be flattened in earlier releases because the image subdirectory isn't respected, however it is not flattened if you use the latest prerelease:

Image
I did download it. It does generally preserve the structure. But there are some cases (maybe deeper subdirs?) where it is somehow flattened. Here is an example:
https://comics.8muses.com/comics/album/Incase-Comics -> https://comics.8muses.com/comics/album/ ... mics/Comic -> https://comics.8muses.com/comics/album/ ... omic/Alfie
The last directory (Alfie) contains the directories Chapter 1-3, Chapter 4-6 etc...
These directories are all created in:
Incase Comics\1 - Comic
insted of:
Incase Comics\1 - Comic\Alfie
and are named AlfieChapter 1-3, AlfieChapter 4-6 etc...

Additionally, I just noticed, there is a deeper directory:
https://comics.8muses.com/comics/album/ ... fie/Extras
that contains 2 directories that, when saved, are named:
AlfieExtrasCharacter Studies
AlfieExtrasOmake
and also some images that, again, are not saved as in the 1st example

It is not as a big problem, I am just mentioning it in case it has an easy solution

User avatar
Squidy
Site Admin
Posts: 1079
Joined: Fri Mar 10, 2017 9:28 pm
Contact:

Re: 8muses - download subdirectories

Post by Squidy » Mon Feb 21, 2022 10:12 am

Thank you for being so thorough. I was able to reproduce the issues, and they should be fixed now with the latest module update.

The directories were getting flattened because the directory separator was getting sanitized away.
I'm the admin and developer of HDoujin Downloader.

dartaron
Posts: 5
Joined: Thu Feb 10, 2022 1:07 pm

Re: 8muses - download subdirectories

Post by dartaron » Sat Feb 26, 2022 10:08 am

Glad I could help you pinpoint the problem.

Thanks for the update!

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests