Maybe the reason is that as mentioned Noir is very much a "love it or hate" type of show, which is true for Madlax, too but not to such a great extent.
I'll have to disagree with you on that issue.

If we compare reactions of those who have seen both series completely, we'll see that unlike Noir, Madlax nearly always leaves a good impression upon the completion. However, while "good impression" of Noir sounds somewhat like "Whoa, that's, like, the coolest anime I've ever watched, I wanna rewatch it and then get some more!", with Madlax it's usually more like "Wow, that's a cool anime. Is there some more of that? Not yet? Ah, whatever... What's next on the list?"
In exchange, in roughly a half of the cases, Noir is remembered like "Oh, that one? I've slept right through most of the episodes and hated the stupid ending." Madlax, on the other hand is mostly reviewed negatively by those who haven't got past the first 8 eps.

My assumptions are based upon observations of my younger brother, whom I managed to convince to watch every Bee Train work we have in the house despite his often-proclaimed programme to start an Anti-BT League, and myself.
OK, I am biased because I don't liek Margaret but I would venture that most viewers share your sentiments.
I can only say for myself, but for me, gradual caring less and less about Margaret started after I understood that she is NOT Kirika v2.0.

Though, my opinion of her rised enormously after that final dialogue with Madlax and Laetitia. The second time I've watched it, I totally cried like a wee lad. %)
I am not so sure about that, some of the "stand alone" or mission of the week episodes did not have too interesting stories but were crucial for character development. So not being interested in the characters seems like a more obvioun hint one should not bother watching any more of it to me.
That advice was, again, based on the observations I've made up to that point in time.

Back then, I didn't set the plot and the characters much apart. %)
The fanservice suggestion is there, whether the anime fully assumes it or not... 
You sure are persistent in reminding us of fanservice in the series. %)
Anyway, I don't dislike Madlax at all, but I don't understand why would someone dislike Margaret?
I mean she's small, cute, orphaned and amnesiac; and she acts funny in this almost autistic sort of way that's just so interesting to watch! She just looks so overly helpless and vulnerable that it feels almost cruel not to care for her! ^^;
Yes, she is cute and autistic, but in my view, she is also "empty". Meaning that she hasn't got anything burning from inside (which is also supported by the plot, wink-wink), only the Gift which is not a personality trait, either. Her amnesia does not torment her a little bit, she just wants to find out about her past - out of pure academic interest, as it seemed to me at some point! This makes her different from Kirika, who was actively suffering all along from her loss of memory.
But Madlax is a entirely different story. Much like Kirika, she woke up one day with a gun in her hands and no memories and had to make a living with it. She is has a stronger will and is generally more detatched than Kirika, which makes her experience less angst, but still suffer nevertheless. Only that her torment is much subtler that Kirika's. Where the latter usually cries a river, Madlax just gets that special expression in her eyes. Plus, the main theme in the series is IMHO that the ability to pull the trigger is inseparatable from the idealistic kindness towards everyone around.

There is a limit on the times a characters can stare blankly into the distance and utter something totally irrelevant after a few seconds long pause and when it's reached the said character starts to get on my nerves. In the case of Margaret the limit was reached probably as early as the fourth episode.
Well, those limits are easily stretched... I mean, Rei Ayanami stares into the distance all way long in Eva and still enjoys attention of the hugest single anime character fandom in the known universe. %)