this is a question with no right or wrong answer...however, i think it never was a story for little children, just as someone here already said, it starts with double murder. a critic just wrote about GOF in the newpaper here, and his words were something like "i started reading the book expecting a children's book. what i found was a thriller, and a bloody good one that i could not stop reading until i'd finished it." and i definitely agree with him, except that when GOF first came i didn't expect a children's book as i'd read PS 48 times, COS 25 times, POA 15 times by then and was sort of familiar with the books already, you might say.
the harry potter books may have started as a story that kids can read, but after the 1st and 2nd book the story gets way darker (ok it's actually dark all along, but it becomes both scarier, more obvious and so on in the later books), and i'd say it's no longer a children's book. the 4th book is definitely not. we're already dealing with pretty heavy and pretty dark stuff, but ALSO other stuff that kids can't understand yet, stuff that you need to be older to understand and take in and see what it really means.
what bugs me is how harry potter everywhere is classified as a children's book, even though it's actually not! i mean, just because the story is about a child/teenage boy, it doesn't mean that it's necessarily MEANT for children. but for the world that seems to be kinda hard to handle. same thing with the movies, they've had to make them way too bright and child-friendly, way brighter and child-frendlier than the story actually is, just because everyone thinks "oh hey yeah i've heard about those HP books/movies, they're for children and children like them, i'll take my little jake to see that one!". at least the 1st and 2nd movie were way too "childish". the 3rd one was better, and i have big hopes for this 4th one. (CAN'T WAIT!!!)
i'm not saying a child can't read harry potter. i was 11 myself when the 1st book came, and i read it and no harm done. but i think i've been in a good age in reading them, as i've sort of "grown with the books" as they got darker and scarier. i mean i was 12 when the 2nd book came, then 13 when the 3rd book came, 14 when the 4th book came, 15 when the 5th book came and then 16 when the 6th book came, so that's been good. now, children can read them allright, but they won't understand all the things and all the emotions and everything that happens that belongs to the young adult / adult -world.
i know POA, GOF, OOTP and HBP aren't books i'd place in the hands of a, say 8 year-old kid of mine.