Forum - View topic
Encyclopedia API


Goto page 1, 2, 3, 4  Next

Anime News Network Forum Index -> Site-related -> Encyclopedia
View previous topic :: View next topic  
Author Message
Dan42
Chief Encyclopedist


Joined: 02 Jan 2002
Posts: 3223
Location: Montreal

PostPosted: Mon Feb 07, 2011 1:40 am Reply with quote
The idea of a public API to access the Encyclopedia data is something I get asked about once in a while, and really I've always thought it would be nice to have. I kept thinking I would build an API into Encyc 2.0 but I've already postponed that for so long... Anime cry

However I just received two emails in quick succession about this topic, so I figured I'd raise the issue publicly and ask those interested: what would you like to see in an Encyclopedia API? How would you like to use it? Please do go into technical details if you care to.
Back to top
View user's profile Send private message Visit poster's website AIM Address My Anime My Manga
herby



Joined: 28 Jan 2010
Posts: 7
Location: Slovakia

PostPosted: Mon Feb 07, 2011 11:29 am Reply with quote
Mainly, I'd like the API to be usable enough to build a facebook app; showing which anime one finished watching / is watching, what are its favourites, send a recommendations to friends, etc.

This means it should have a search based on title(s), and ability to get details. I think majority of the details shown on encyclopaedia pages would be nice to have in the API (including the ranks in the charts).

Of course there can be more use cases, but I am concerned mainly on this one (not talking about integration with myanime yet / I think it would scare users away if username on ANN would be required; not to mention there should be an API for that as well if it should work).

And there's one more use case, which is apparent ("eat your own dog food" principle): the encyclopaedia itself.

It may help in API design if the encyclopaedia itself would be considered an as an app based on top of the API.
Back to top
View user's profile Send private message ICQ Number My Anime My Manga
gerjomarty



Joined: 21 Apr 2007
Posts: 3
Location: Cambridge, UK

PostPosted: Mon Feb 07, 2011 11:55 am Reply with quote
I'd be very interested in utilising an API to the encyclopedia, mostly for use in mobile applications that can use the data along with other, similar sources/APIs. The more data the better. Smile
Back to top
View user's profile Send private message Visit poster's website
herby



Joined: 28 Jan 2010
Posts: 7
Location: Slovakia

PostPosted: Mon Feb 07, 2011 12:30 pm Reply with quote
As for implementation, I would draw some inspiration from the fb graph api: make each entity (manga, anime, person) an object with its own url (maybe http://graph.animenewsnetwork.com/​id, maybe else), having connectors on each page (http://graph.animenewsnetwork.com/​id/​crew for animes, http://.../id/picture for any entity, http://.../id/works for persons, http://.../id/related (sequel, retelling, etc.) and the like). All pages would return json structure containing the information.

Other niceties could be to allow numerical id as well as common latinized slash-separated name as alternate ids for the same object; and the graph uri for "meta-objects" as well: genres, themes, charts, types (anime, manga, person, genre, ...) to be able to get info but enumerate as well through (for example) /list connector.

Batching and filtering is good to think up in advance, it can save bandwidth.

For example, id can be list of comma-separated ids (saves doing lot of separate requires).

As for filtering, there are two things: first, filter only relevant elements (like, from .../crew I only want to see the director(s)), and the second stage, filtering only required fields. Another complication, which may be included in filtering section, is the multillinguality - there's a lot of titles in various languages, but even if we omit that, there is "official" title which is sometimes shown as english, sometime as japanese in romaji, though there is japanese in kanji and kana as well; this holds for persons, too.

For filtering, I'd use parameters of the request.
Since structure of the output will be always well defined, one could use the parts of the structure itself with some operators to do the filtering, like:
http://..../anime,manga/list?[].data[].includes(genre)=slice+of+life&[].data[].includes(themes)=rural
finds all animes and mangas with genre "slice of life" and theme "rural".
http://.../anime/list?[].data[].matches(director.name.lat)=Miyazaki is obvious (lat is the proposal for "common latinized form, whatever the original is", which can save lot of troubles)
http://.../id/?[].fields(allow)=id,title.lat,picture,readableid,rank.bayes only gets listed fields
http://.../anime/list?[].data[].matches(title.lat,nocase)=suzumiya&[].data[].fields(allow)=id,picture,title&[].data[].title.fields(prefer,2)=jp-ro,en,lat,jp gets id, picture and at most two titles of all suzumiya animes, while preference of titles is as listed above

The syntax of the parameters is always the same: path.to.element.operator(details)=argument
I assumed the result is json array of objects, each giving info on one piece, that's why it always began with [] (meaning "any element in the array") and then continued with usual dot notation (so the [] was nothing special, part of the "path.to.element").

Rules as to where to OR and where to AND is to be thought of, but it seems the "OR the one with the same name, AND the aforementioned OR groups" could work.
Other operators could see its way there like sort(comma-separated fields), range(from,to) for fields, limit(to,[from]) for occurences, and maybe also flatten(part-of-initial-path) to select only array of some subfields (ignoring the harness it was put into) as the result (like http://.../anime/?flatten=[].data[]&matches(director.name.lat)=Miyazaki) puts away the [{type:"anime", data: [ ... ]}] and only includes [ ... ] as a result; if it was anime,manga it would join both datas).
Back to top
View user's profile Send private message ICQ Number My Anime My Manga
Dessa
Baka RangerBaka Ranger


Joined: 14 Jul 2004
Posts: 2382

PostPosted: Mon Feb 07, 2011 2:04 pm Reply with quote
For those of us who are dumb, what is an API, what does it do, and what is it used for?
Back to top
View user's profile Send private message My Anime
Shiroi Hane
Encyclopedia Editor


Joined: 25 Oct 2003
Posts: 5980
Location: Wales

PostPosted: Mon Feb 07, 2011 2:17 pm Reply with quote
http://en.wikipedia.org/​wiki/​Application_programm​ing_interface

It allows you to programmatically access the encyclopedia data. Currently, if another site wanted to display data from the encyclopedia they would have to either copy it manually or scrape the pages. API access would be less resource hungry and would also allow you to do your own processing on the raw data if you want to produce some sort of interpretation that the encyclopedia currently doesn't.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger ICQ Number My Anime My Manga
JackPerry



Joined: 07 Feb 2011
Posts: 1

PostPosted: Mon Feb 07, 2011 2:34 pm Reply with quote
Hey Dan,

I was one of the ones that emailed you. Thank you for a quick response. Now, my main idea for using the API stemmed from my idea to build a social anime recommendation application based on your friends or even strangers who share the same anime preferences as you do. I looked around and it seemed the only viable solution to grab anime data would be to use MyAnimeList. Unfortunately, their API isn't so great and reportedly, slow. Besides that, some of MAL's data (descriptions, etc) claim ANN as the source. From that I realized, why not GO to the source?

As far as technical reasons go, I wouldn't require a lot. As long as it is accessible, and fast (this is the big one) I'm sure I could work with it. I do work in PHP, Ruby, Python, and sometimes Objective-C for iOS development so as long as it is clean and well structured there should be no problems.
Back to top
View user's profile Send private message
herby



Joined: 28 Jan 2010
Posts: 7
Location: Slovakia

PostPosted: Mon Mar 21, 2011 2:57 am Reply with quote
Dan? Any progress?
Back to top
View user's profile Send private message ICQ Number My Anime My Manga
herby



Joined: 28 Jan 2010
Posts: 7
Location: Slovakia

PostPosted: Mon Apr 11, 2011 4:53 am Reply with quote
BTW, FB Graph API has batching ability which uses thing called JsonPath, so it may probably be reused in ANN API, not to reinvent the wheel and to reuse existing thing...
Back to top
View user's profile Send private message ICQ Number My Anime My Manga
akhild



Joined: 03 Mar 2010
Posts: 1

PostPosted: Sat Apr 23, 2011 3:41 pm Reply with quote
Hey, I joined recently, and wanted to use this type of app to keep records for my local application, and making relations to fb posts too. So has there been any update on this?
If not, I would like to help in the same.

Second reason for this API would be able to get archived information like imdb API, which can include genre, comments and other anime specific details, without loading the url and manually extracting the data based on the page's html output
Back to top
View user's profile Send private message My Anime My Manga
herby



Joined: 28 Jan 2010
Posts: 7
Location: Slovakia

PostPosted: Thu May 19, 2011 9:16 am Reply with quote
Text search - an API which by supplying string returns all works as well as people whose names / titles match (maybe with some degree of fuzziness to clear misspells). Useful for lots of ajax applications.
Back to top
View user's profile Send private message ICQ Number My Anime My Manga
Dan42
Chief Encyclopedist


Joined: 02 Jan 2002
Posts: 3223
Location: Montreal

PostPosted: Tue Sep 06, 2011 9:02 am Reply with quote
Hi everyone,

Sorry to keep you waiting. I did read the responses to my opening post but then got distracted by other ANN work.

But I finally bit the bullet and started coding. First I modified the Encyclopedia reports to provide an option for XML output. Go to any of the reports and you'll see a display as xml link. Some of those reports can be customized through url parameters, turning them into a rough kind of API. For example the Anime/Manga List API allows to restrict results by adding a "name" parameter to the url, like this.

At first I was going to make such a report to output anime details/credits/etc, but then realized it would be simpler to reuse the existing codebase, so I made a separate API to retrieve anime details. The API is a simple REST call to http://cdn.animenewsnetwork.com/encyclopedia/api.xml?k=v&k=v&...
where k can be
* anime
* manga
* title (either anime or manga)
and v can be
* a numeric id
* a slash-separated list of ids
* a string to search for, prefixed with ~ (limited to 10 results)

For example:
http://cdn.animenewsnetwork.com/​encyclopedia/​api.​xml?​anime=​42
http://cdn.animenewsnetwork.com/​encyclopedia/​api.​xml?​anime=​123/​234/​345&​manga=​1595
http://cdn.animenewsnetwork.com/​encyclopedia/​api.​xml?​manga=​~​bob
http://cdn.animenewsnetwork.com/​encyclopedia/​api.​xml?​title=​~​jin

I'm open to suggestions on how to improve and expand the API, but please keep in mind that we can't turn this into a full data-dump of the entire Encyclopedia. And unlike Facebook/Google/etc our resources are limited, so we can't power other people's ajax applications for free.

Oh, and speaking of limited resources... please avoid hammering the API.
Back to top
View user's profile Send private message Visit poster's website AIM Address My Anime My Manga
AppuriTech



Joined: 16 Sep 2011
Posts: 1

PostPosted: Fri Sep 16, 2011 3:03 am Reply with quote
Hi Dan42,

I'm an Android app developer and anime fan, and would be interested in using your API to make an AnimeNewsNetwork app, if ANN approves of it, of-course.

It would be useful for viewing front-page news, looking up anime/manga on the go, etc. And generally avoiding loading full ANN.com desktop version on a mobile phone.. (especially if you have 2G)

Quote:
unlike Facebook/Google/etc our resources are limited, so we can't power other people's ajax applications for free.


If you're worried about excessive use of the API, the app could be subscriber-only (required to login) or make use of ann advertisements.

This may have better as an email, but you probably consider your inbox alot of spam, and also this way I can hear if ANN users would use an Android app.

Let me know,
Thanks.
Back to top
View user's profile Send private message Visit poster's website
herby



Joined: 28 Jan 2010
Posts: 7
Location: Slovakia

PostPosted: Tue Dec 13, 2011 5:21 am Reply with quote
It is not working. I click on the link, it redirects me to cdn.animenewsnetwork.co.uk and there it fails with "Oops! Google Chrome could not find cdn.animenewsnetwork.co.uk". If i use .com in api url, it redirects me to img.animenewsnetwork.co.uk and fails the same way.
Back to top
View user's profile Send private message ICQ Number My Anime My Manga
Shiroi Hane
Encyclopedia Editor


Joined: 25 Oct 2003
Posts: 5980
Location: Wales

PostPosted: Tue Dec 13, 2011 7:24 am Reply with quote
I'm sure I've accessed the API page before but yes - it keeps redirecting me to non-existent UK servers, even after invoking redirect_to_best_locale=off
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger ICQ Number My Anime My Manga
Display posts from previous:   
Reply to topic    Anime News Network Forum Index -> Site-related -> Encyclopedia All times are GMT - 5 Hours
Goto page 1, 2, 3, 4  Next
Page 1 of 4

 


Powered by phpBB © 2001, 2005 phpBB Group