Yes, I tend to be wary of start-up presses too (though in my case also, the matter is moot, since I'm no longer marketing myself to publishers). It doesn't matter so much with e-zines, because if an e-zine goes out of business after the first issue, you've had your payment for the story (or your resume credit), and all's fine. But having your press go out of business soon after it publishes your novel is a bummer. Poor
Susan R. Matthews, for example, just had a novel published with Meisha Merlin last year; now it's out of print, and the revised reprints of her novels that Meisha Merlin had planned won't come to fruition. Not that Meisha Merlin was a recent start-up, but the emotional impact is the same on authors when presses fail.