Forum - View topic
Release Scraper Completely Broken Unable to Add Releases


Goto page Previous  1, 2, 3  Next

Anime News Network Forum Index -> Site-related -> Bugs & Technical Questions
View previous topic :: View next topic  
Author Message
Tenbinzan



Joined: 18 Dec 2006
Posts: 22
PostPosted: Sat Jul 20, 2019 11:42 am Reply with quote
Figured I'd bump this...

I was trying to add some eBooks to the Encyclopedia today and kept coming across the "could not fetch page" error with Amazon/B&N links. Doesn't matter what I try. Sad
Back to top
View user's profile Send private message
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1006
Location: Minnesota, USA
PostPosted: Tue Jul 23, 2019 5:35 am Reply with quote
Tenbinzan wrote:
Figured I'd bump this...

I was trying to add some eBooks to the Encyclopedia today and kept coming across the "could not fetch page" error with Amazon/B&N links. Doesn't matter what I try. Sad
The scraper has been broken for over 2 years, so it's nothing new sadly, which is very frustrating given just how valuable the encyclopedia is for release history, it's been my #1 resource for release history for as long as I've been collecting, from the contents (episodes, extras, etc.), MSRP, release date, etc. Really wish it would get fixed already but I'm not holding my breath.
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
Tempest
I Run this place.
ANN Publisher & CEO


Joined: 29 Dec 2001
Posts: 10129
Location: Do not message me for support.
PostPosted: Wed Jul 24, 2019 11:17 am Reply with quote
I apologize for the wait. Unfortunately we don't have a timeline to fix this right now.
Back to top
View user's profile Send private message Send e-mail My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1006
Location: Minnesota, USA
PostPosted: Wed Jun 24, 2020 4:11 am Reply with quote
It's been almost a year, so I felt like bumping this to ask if there's any chance at least one or two websites will get fixed so stuff can be added again. The scraper has now been broken for 3 years, that's a lot of releases that are straight up missing, and to make matters worse is how RightStuf delists OOP releases often, which makes adding stuff that was exclusive there harder if it's after it sold out.
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
horseradish
Subscriber



Joined: 27 Oct 2015
Posts: 569
Location: Bay Area
PostPosted: Wed Jun 24, 2020 1:17 pm Reply with quote
I initially kept a list of new releases to add in the future when the scraper got fixed, but quickly realized how hilariously futile it was due to volume. Anime dazed It was a very useful feature...still is, but getting more and more outdated.
Back to top
View user's profile Send private message My Anime My Manga
horseradish
Subscriber



Joined: 27 Oct 2015
Posts: 569
Location: Bay Area
PostPosted: Wed Jun 24, 2020 2:19 pm Reply with quote
I've been wondering this for a while...Would it be feasible to grant maybe a handful of members the ability to only manually add and edit releases? They would otherwise make basic changes that any regular contributor can. After all, certain encyclopedic tasks can only be done by some Encyclopedia Editors, not all. If there was a quick and basic guide to releases, perhaps this would be possible? BigOnAnime seems eager to do it and has continued to show interest in this endeavor. Not me though, since I'm too busy with work. Maybe with at least one dedicated person, it would lessen the backlog?
Back to top
View user's profile Send private message My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1006
Location: Minnesota, USA
PostPosted: Wed Jun 24, 2020 3:43 pm Reply with quote
I've just kept WTK's release date lists handy for this sort of stuff. That's how I would add stuff, I would go down the list to see if anything was missing, though it could still be time-consuming. Shortly before the scraper broke, I was apparently the only person bothering to add releases anymore as nobody else was really doing it. Even the person doing Shelf Life wouldn't add stuff themselves and would note how stuff wasn't added.
https://drive.google.com/file/d/1wFMrs5wCA-CfdyikHGFGLAq4FkkbpKqZ/view
https://forum.dvdtalk.com/anime-talk/641435-upcoming-u-s-anime-releases-2018-edition.html
https://forum.dvdtalk.com/anime-talk/645354-upcoming-u-s-anime-releases-2019-edition.html
https://fandompost.vbulletin.net/forum/anime-manga-discussions/us-blu-ray-dvd-and-simulcast-industry-news/1205-u-s-anime-release-dates-unreleased-licenses-forum-resource-links-thread

Just to note though, I only ever really added home video releases, not manga, novels, light novels, nor their eBook versions. Someone else would have to handle that task of adding all those missing releases.
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
Triltaison



Joined: 03 Jul 2011
Posts: 480
PostPosted: Thu Jul 02, 2020 2:52 pm Reply with quote
For what it's worth I was helping you submit too, BigOnAnime. I was doing mostly old stuff that was hard to find information on like those weird $1 DVD releases by Digiview that were the only way to watch some anime, old comic and VHS releases that were the only format some series received, Walmart-only releases like the Attack on Titan DVD only sets, out of print manga, and other oddities. I also tried to add anything that I purchased for my own collection and couldn't find listed in the database.

If we're ever available to submit again, I'd be happy to help you out. I really hate how out of date everything is now... this used to be a fantastic resource.
Back to top
View user's profile Send private message
Tempest
I Run this place.
ANN Publisher & CEO


Joined: 29 Dec 2001
Posts: 10129
Location: Do not message me for support.
PostPosted: Sat Jul 04, 2020 6:06 pm Reply with quote
I'm going to be looking into this in the coming days and see what can be done.

-t
Back to top
View user's profile Send private message Send e-mail My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1006
Location: Minnesota, USA
PostPosted: Tue Aug 04, 2020 3:13 am Reply with quote
It's been a month, any updates on what can be done about the release scraper being broken for 3 years at this point?
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
Chibi Shoujo



Joined: 11 Dec 2006
Posts: 97
Location: Wisconsin, USA
PostPosted: Sat Sep 12, 2020 7:33 am Reply with quote
horseradish wrote:
I've been wondering this for a while...Would it be feasible to grant maybe a handful of members the ability to only manually add and edit releases? They would otherwise make basic changes that any regular contributor can. After all, certain encyclopedic tasks can only be done by some Encyclopedia Editors, not all. If there was a quick and basic guide to releases, perhaps this would be possible? BigOnAnime seems eager to do it and has continued to show interest in this endeavor. Not me though, since I'm too busy with work. Maybe with at least one dedicated person, it would lessen the backlog?


I do think this would be beneficial, especially to help the Encyclopedia Editors focus on other issues. Personally, I'd volunteer and love to help out with the manga side of things since that's the part I care about most on this site and I'm usually checking new U.S. releases at least once a month. (Plus side of being an avid collector? Laughing ) I'd even love to be able to go through the manga index and utilize the legend for bolding licensed manga, since that's extremely outdated too.
Back to top
View user's profile Send private message My Anime My Manga
v_i
Encyclopedia Editor


Joined: 23 Dec 2018
Posts: 6
PostPosted: Mon Feb 22, 2021 8:54 pm Reply with quote
I would be willing to manually input a bunch of releases, if it were so allowed. The ANN release encyclopedia is amazing, and super helpful, but it becomes less and less relevant with each passing release that's not in the database. I want to help fix it, but can't, as any and submission must go through the scrapers.
Back to top
View user's profile Send private message
Tempest
I Run this place.
ANN Publisher & CEO


Joined: 29 Dec 2001
Posts: 10129
Location: Do not message me for support.
PostPosted: Mon May 03, 2021 4:41 pm Reply with quote
Good news. Dan is working on this now.
Back to top
View user's profile Send private message Send e-mail My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1006
Location: Minnesota, USA
PostPosted: Thu May 13, 2021 11:36 pm Reply with quote
There is hope for the release scraper? I've been waiting for 4 years for this. It would still be nice though if select users had the ability to manually add certain releases and edit them (Ex: Linking to multiple entries) so encyclopedia editors can focus more on more pressing matters (the release corrections thread is also dead). I keep finding numerous things that need correcting.

Also, can Dan look into adding a 4K section for releases? While it's pretty clear at this point the format is never going to really take off for anime, there's been a few releases that have hit here so far, both imports and domestic releases. There should be a proper way for them to be cataloged into the site.
https://www.rightstufanime.com/live-action/false?keywords=4K
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1006
Location: Minnesota, USA
PostPosted: Sun May 16, 2021 7:37 pm Reply with quote
So I noticed recently the RightStuf links were black instead of gray and decided to give adding releases a try, and it worked, I can't believe it actually worked. Note that only RightStuf URLs work at the moment. I just added these.
release#36613
release#36614 (This needs to also be linked to the movie and OVA)
release#36619
release#36615
release#36622

Also, seems it can't organize the Blu-ray region at this time, these should be under (Region A) but they aren't. Descriptions aren't being pulled from the site, though I can't remember how long it took years back when the scraper was fully functional. Baby steps, but I can't thank Dan enough for getting at least this to work.

Something else to point out, I wasn't able to add this, I got back an error saying the UPC wasn't from a known distributor.
https://www.rightstufanime.com/Cardcaptor-Sakura-Complete-Series-Standard-Edition-DVD

[Edit]IIRC this was an issue before, but I tried to add a combo pack, and it added it as a DVD instead of a combo pack.
release#36617

DVDs have no issues to speak of at the moment.
release#36616

[Edit 2]I was unable to add this because it didn't find the UPC.
https://www.rightstufanime.com/Yuki-Yuna-is-a-Hero-Complete-Edition-Blu-ray

Also adding URLs to releases missing them worked. This was manually added and had no URLs until now.
release#35705

Tested adding manga and light novels and these went fine.
release#36620
release#36621

[Edit 3]Want to lastly note, Rakuten should probably be removed from the list seeing as it shut down.
https://techcrunch.com/2020/07/30/rakuten-is-shuttering-the-online-shop-formerly-known-as-buy-com/

To give additional options so there's more than 4, maybe adding the Sentai Filmworks Shop and FUNimation Shop to the list could work?
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
Display posts from previous:   
Reply to topic    Anime News Network Forum Index -> Site-related -> Bugs & Technical Questions All times are GMT - 5 Hours
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 


Powered by phpBB © 2001, 2005 phpBB Group