Most anime fans are familiar with the subgenre colloquially named "Cute Girls Doing Cute Things". The Case of Hana & Alice, though, is an interesting variant; Kooky Girls Doing Kooky Things. Admittedly, their kookiness is cute, but this feels almost like a by-product. What makes Hana and Alice, and Hana & Alice, appealing is the characters' natural oddness, which leads them into absurd adventures that they see as quite reasonable.

This feature film divides into two halves. In the first, “Alice” – short for Arisugawa, the girl's family name – arrives in what her single mum calls the boonies. Hana hardly appears in these scenes; it's Alice we follow as she learns about a legend in her new school, about a boy called Yuda (“Judas”), who somehow had four wives and who was then mysteriously murdered. This tale has spawned a teen cult in Alice's class, whose members draw magic seals and perform exorcisms to contain Yuda's spirit.

While the set-up feels a bit like Another, it's no spoiler to say Hana & Alice isn't that kind of story; it's about kooks, not spooks. The characters are middle-schoolers (Alice is 14), but often seem like preteen children in their behaviour. No-one says chunibyou – the Japanese slang for undeveloped middle-schoolers, often wrapped in fantasies – but the idea underlies the film. It's not just the teenagers. Alice's mother, a writer, seems as chunibyou as Alice herself; for example, she tries to match-make two of Alice's teachers within seconds of meeting them.

Hana & Alice, though, doesn't follow the film rulebook for oddballs. Unlike many youngster-moves-to-new-place stories, Alice has no great problem fitting in to her new neighborhood. She's weirded out by the class cultists for a bit, but is soon dispatching a boy bully with a stomp on his privates and a punch in his mouth (“That'll teach you to pick on a transfer student!”). She's presented as neither average nor remarkable. In the early scenes, Alice is actually the “straight” character, looking on bemused as her classmates perform their wacky ceremonies. They're led by a long-haired girl who's the class witch/priest/possessed vessel… though she soon flips from scary to sympathetic.

These scenes are diverting, often charming, but they can also feel annoyingly shapeless, without a clear centre for our attention. Thankfully, that changes halfway through the film when Alice meets Hana, the shut-in (hikkikomori) girl next door. By this point, Alice is committed to solving the mystery of Yuda. Hana is at its heart; could she even be Yuda's killer? While Hana leaves that question hanging, she quickly takes charge, sending Alice on a strange journey into Tokyo while secretly following behind. The rest of the film follows their misadventures, a comedy of errors, digressions and mistaken identities that somehow leads them to their goal.

It's this part of the film that really works, partly by discarding much of what came before – the support cast from the first half just disappears. The girls' torturous quest is funny and engaging, thanks largely to their constant confusion as events spin out of their control, though they rarely seem in serious trouble. Their slow recognition that they're kindred spirits, who can benefit and learn from each other's differences, is tender and free of schmaltz. It's not a kind of story we often see in anime. The girls' comic journey round Tokyo is slightly reminiscent of the tramps' urban odyssey in Tokyo Godfathers, but with none of that film's danger.

Hana & Alice doesn't look like many other anime either. It's a rotoscoped production, filmed in live action and painted over in the manner of the 2013 serial Flowers of Evil (though the two anime don't seem to share any artists). Rotoscoping usually has problems, even if you accept it as a legitimate kind of animation. In Hana & Alice, there are many points when the movement just looks wrong. In the very first scene, Alice falls from a window, to be caught by a man on the ground; this looks okay, with a real sense of weight. Then Alice jumps out of the man's arms, in far less convincing fashion. In animation, weight is one of the hardest things to convey; rotoscoping often loses it between the lines.

Throughout the film, impressively convincing movement alternates with clunky shots. On the plus side, there are lots of believably cute interactions between girls, giving substance to even minor characters through how they sit or fidget. Facial expressions range from crudely overamped (Alice throwing a fit) to poignantly subtle (Hana confessing her past). Alice looks most convincing when she's dancing or running; at best, her running feels like first-grade traditional animation. When Alice chases a car, for example, there are some excellent, dynamic shots of her sprinting up a pavement – and then a weaker shot where her legs are just an affectless tracing.

Perhaps the film's best visual is one towards the end. It's an upward shot of Hana dancing under the stars, a troubled girl lost in unexpected peace. It's truly moving, redolent of a pivotal movement in Flowers of Evil, which again showed two characters in a midnight "dance." Rotoscoping seems suited to such stylized movements and angles in anime; but even so they don't always work. For example, Hana & Alice has the annoying habit of tipping the camera ninety degrees sideways at arbitrary moments. It feels pointlessly precious, as if someone was worried the film didn't feel "indie" enough.

The backgrounds are more consistent than the characters, though some have a glazed, almost congealed feel. But it's fun to spot the live-action Japanese locations under the drawings. Alice meets with her divorcee dad in the town of Kamakura, and she and Hana go to what seems to be the area around Makuhari Messe in Chiba, the venue for the Tokyo Game Show and Niconico Douga.

The film's rotoscoped format, coupled with the lack of fantasy content, brings an old chestnut to mind; why animate such a story? While there are various rationales (some budgetary), the most pertinent may be that it fits the subject; a tale of imaginative youngsters who look at the world and see more than mundane reality. The same was true of Flowers of Evil, and traditionally animated works such as Whisper of the Heart.

The director of Hana & Alice is an outsider to anime, Shunji Iwai, who's worked for more than two decades in live-action. Many of his films have teen protagonists, including his best-known work internationally, All About Lily Chou-Chou (2004), which was a darker portrait of adolescence. The closest Iwai previously came to anime was his lead role in Hideaki Anno's excellent live-action Ritual Day (Shiki-Jitsu, 2000). In it, Iwai played a burned-out director, a stand-in for Anno after End of Evangelion, when Anno claimed he had no ideas left to animate.

The Case of Hana & Alice is a prequel to a film Iwai made about the same characters in 2004, which was just called Hana & Alice. Thanks to animation, Iwai could bring back his original actors without ageing the characters. Alice is voiced in Japanese by Yū Aoi, Megumi in the recent live-action Rurouni Kenshin films. The deeper-voiced Hana is played by Anne Suzuki, who was the boy hero in Steamboy and Natsuki in the South Korean film of Initial D. Both actors are excellent, still sounding like convincing teens. (The one obvious vocal mismatch in the film is the septuagenarian Sei Hiraizumi, who sounds much too old voicing Alice's father.)

Despite its aesthetic issues and a meandering start, Hana & Alice is a highly likable film about hugely likable teens. It's well worth catching on the big screen, should you have the chance. I first saw it in a cinema (at last year's Annecy festival), and was struck by how much it felt enhanced by the audience, laughing along with the girls' misadventures. The film may well remain an anomaly, a one-off; it's hard to see it having many imitators. But as its shaggy-dog story reminds us, being different is good.

The film has the following screenings scheduled in Belfast, London and New York:

- At the Queen's Film Theatre in Belfast on 21 February at the Queen's Film Theatre at 6:30 p.m., as part of the Takeover 2016 festival.

- As part of the ASIA HOUSE FILM FESTIVAL which takes place from 22 February to 5 March at London venues. The Case of Hana & Alice will be screened on 27 February at Regent Street Cinema at 3 p.m.

- There will be three screenings at the New York International Children's Film's Festival, which runs from February 26 to March 20. The Case of Hana & Alice will be screened at the Bow Tie Chelsea Cinemas on February 28 at 1:30 p.m., and on March 13 at 4:15 p.m. The film will also be screened the film at the IFC Center on March 20 at 12:30 p.m.