Scrape Photos From Irasutoya dot Com

Irasutoya is a Japanese stock photo website, using a cute, cartoon style for their photos. I am a big fan of Irasutoya. I would like to download the whole entire website. I have tried this before, via the 'wget' program, on a third party search feature for the website, to no success. The website indexes all of it's files by the appropriate categories (i.e. https://www.irasutoya.com/2012 (you could also select the year the photo was published, like I've done here)) The website indexes it's photos by giving a grid of 8x3 of low quality thumbnails, and once a photo is clicked it takes the visitor to another page with another low quality thumbnail and the photo's description, which needs to be clicked on to attain the high quality version of the file. This then takes the user to a random blogspot domain, with a seemingly random page accessed on the blogspot website. To display more photos for the chosen category, one needs to click on the "次のページ" button. This is where my problem lies; I lack the programming skills to scrape this website -- this seems too complex of a procedure for a script-kiddie like me to solve, even with online "guides", due to how one comes about the high quality image. What does Lain personally suggest?

>>27007 The website doesn't require any javascript so wget is the right tool. The issue is just that wget will not follow links to other websites by default, otherwise you will download the whole internet. Use -H to allow going to other websites and -D to specific which website is allowed. I also added -A to only download html and png because you don't care about other stuff on the site. wget -r -np -H -D blogspot.com -A "*.html,*.png" https://www.irasutoya.com/2012/12/blog-post_4860.html Read the manual to see if you can find any other options that will make the command better. https://www.man7.org/linux/man-pages/man1/wget.1.html

>>27007 Python and selenium, I've ripped pdfs that only allow image previews this way before. Use selenium to select and save (for hq see below advice) all images of the class 'boxthumb' and then click the button and repeat that forever. The script will finish/crash when there is no button, ending when all the images have been downloaded. Btw you can get the high quality version of the preview image for saving by removing 's180-c/image_name.png' from the preview url. If you get stuck I can help.

Damn, this thing is asking for a booru. Can't believe no cunt-eyed autist has scraped this soykaf yet.

>>27022 You can put any number there, but it will stop rendering at the highest resolution uploaded to the server once you exceed it, I tried 1000 and 2000. https://1.bp.blogspot.com/-b-y8m6_Z8gI/VrWkfmhHjgI/AAAAAAAA30c/GabeLqOA0VU/s1000/ Gives a 802*894px while on blog page it offers s800: https://1.bp.blogspot.com/-b-y8m6_Z8gI/VrWkfmhHjgI/AAAAAAAA30c/GabeLqOA0VU/s800/saiyuki_sanzouhoushi.png which is 718*800px Your advice does not render highest resolution: https://1.bp.blogspot.com/-b-y8m6_Z8gI/VrWkfmhHjgI/AAAAAAAA30c/GabeLqOA0VU/ is 459*512px Some items have multiple images in them, scraping just thumbnail urls won't do. Also, https://www.irasutoya.com/2015/12/blog-post_402.html Tops at 1500*1117px So a wild guess would be to put a very large number in that field just to be sure.

https://www.irasutoya.com/search/ Is a directory of all entries. Arrow right to navigate earlier pages. Each entry has a <span class="category"> field with tags for given image or group of images. So I guess even if it contains more than one image, you save each image and assign same tags to it, so when searching in a booru interface, listing all tags will fetch all connected images, or if possible to connect them into a group or whatever it was called.

>>27022 I don't see any javascript so selenium is a waste. If you are a programmer (unlike OP), then Python/BeautifulSoup or Ruby/Nokogiri would be the best solution for crawling exactly the files you want and parsing out metadata like tags as well.