I don't have deep working knowledge of these kinds of solution finders, but you might want to look at something that has been formalized and looked at over the past couple of decades, the EM algorithm is a good one to look into:
In terms of your actual issues, this is going to (if your code has been correct) be an issue of looking into the stability of your various scheme.
In terms of Normal distribution fitting, the EM has a lot of implementations to do this including in packages like R and if you can look at the R code for this algorithm, then you can get a much better idea of what's going on and compare it to your own implementation:
The R Project for Statistical Computing
Now a search on the implementation brought up a thread like this and after reading this I am hesitant to offer specific advice, so I will just post the thread which should give you food for thought:
R help - EM algorithm
For a normal model (as in normal PDF), there was a paper that implemented EM in R that uses bootstrapping in combination with the EM: