Results 1 to 13 of 13

Math Help - how to regularize an ill posed problem

  1. #1
    Newbie
    Joined
    Mar 2011
    Posts
    11

    how to regularize an ill posed problem

    Hi,
    This is my first contribution to this forum
    I have some problems to get regression to this equation:

    my goal is to estimate the different a,b ans c knowing u and y.
    What I know are u and y vectors over time. It would not be difficult to compute the different integrals.

    My fist trial was to wright the linear system:



    Like that, the problem seems to be ill posed, in fact the estimation of a, b and c do not converge (the algorithm is unstable). I tried Tikhonov regularization, improved the stability for some parameters but not for all:

    I choosed Q and P ad-hoc.
    I think the illness is due to:
    - the choice of A and b: why should I wright u= and not = or y= or somethink else?
    - The fact that in A there are some signals that are not independent:some signals are the integrals of others.

    How can I regularize this ill problem? or how to wright it as a well posed problem? How to consider the integral relationship? rigorously.

    thanks
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Senior Member
    Joined
    May 2010
    From
    Los Angeles, California
    Posts
    274
    Thanks
    1
    You haven't indicated the regions over which the integrals are to be done. Do you have a reference for this problem?
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Mar 2011
    Posts
    11
    Thank you for answer.
    You haven't indicated the regions over which the integrals are to be done.
    Sorry, you'r right. it's an integration over time so it's between zero and final instant (t)
    Do you have a reference for this problem?
    what do you meen by reference?
    thanks
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Senior Member
    Joined
    May 2010
    From
    Los Angeles, California
    Posts
    274
    Thanks
    1
    The statement is still confusing. In one instance you're performing a single integral of u but in another you're performing a double integral. How are you viewing u both as a function of one and two variables? You wrote b=Ax but neither b, A or x is defined.

    By a reference I mean where did the problem come from. Is it something out of a book or is it something you came up with?
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Newbie
    Joined
    Mar 2011
    Posts
    11
    In one instance you're performing a single integral of u but in another you're performing a double integral. How are you viewing u both as a function of one and two variables?
    This is simply a second order differential equation, like for example position is the integral of velocity and the second integral of acceleration. both simple and double integral are between 0 and t.
    By a reference I mean where did the problem come from. Is it something out of a book or is it something you came up with?
    This equation cames from a real physical model.
    Rigorously, the equation of the model is: u''+a_1 u'+a_2 u+d_1 z'+d_2 z = c_1 y''+c_2 y'
    Physically, derivation is not causal ans causes problem is numerical computing so I integrate que equation twice and obtained:

    u + a_1 \int_0^t u\, \mathrm dt + a_2 \int_0^t \int_0^t u\, \mathrm dt^2 - c_1 y - c_2 \int_0^t y\, \mathrm dt + d_1 \int_0^t z\, \mathrm dt + d_2 \int_0^t \int_0^t z\, \mathrm dt^2 = 0.
    I know u and y but not z, I however know z is quite constant over time so I wrote:
    d_1 \int_0^t z\, \mathrm dt + d_2 \int_0^t \int_0^t z\, \mathrm dt^2=b_1 \int_0^t 1\, \mathrm dt + b_2 \int_0^t \int_0^t 1\ dt^2
    You wrote but neither or is defined.
    My goal is to have a good estimation of a_1, a_2,c_1 and  c_2.
    b_1 and b_2 or d_1 and d_2 are not important for me, however as a regressor estimates all x vector, I think that bias on b would causes bias on other values. so a posed arbitrary a linear equation:
    b= A x
    where
    b=u; A=[-\int_0^t u\, \mathrm dt ,-\int_0^t \int_0^t u\, \mathrm dt^2,- \int_0^t 1\, \mathrm dt ,- \int_0^t \int_0^t 1\, \mathrm dt^2 , y , \int_0^t y\, \mathrm dt ]
    and computed x by least mean square regressor witch gives me an estimation of:
    x=[a_1,a_2,b_1,b_2,c_1,c_2]'
    -------------------------------------------
    an other information that I don't know how to exploit is the fact that:
    a_1/a_2=c_1/c_2=d_1/d_2

    I hope this contribution is little bit clearer than previously.

    Thanks for contribution
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Senior Member
    Joined
    May 2010
    From
    Los Angeles, California
    Posts
    274
    Thanks
    1
    OK, so the original problem was a differential equation, not an integral equation. I can see immediately that you are missing initial conditions on the unknown functions.

    You should probably describe the physical situation that gave rise to this formulation. It doesn't look like you've done it correctly because, in addition to not specifying initial coinditions, you seem to have too many unknown functions and parameters.
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Newbie
    Joined
    Mar 2011
    Posts
    11
    Hi Jones,
    Previous works (not mine) in our company gave this differential equation of a special type of gearbox where
    u is a measured input torque
    y is a measured velocity
    ans z is an unmeasured disturbance torque
    It doesn't matter if I don't know the disturbance but I need to know correctly th parameters.
    I can see immediately that you are missing initial conditions on the unknown functions.
    In automation, initial conditions of signals and their different derivatives are often supposed null.
    You can view the transformation that I did is to apply a double integrator filter to all inputs and outputs:
    Y(s)= H_1(s)*U(s)+H_2(s)*Z(s)   so   F(S)*Y(s)=F(s)*H_1(s)*U(s)+F(s)*H_2(s)*U(s)
    where F(s)= 1/s^2
    I will give you tomorrow a set of data and resultsview better the problem.

    Thanks
    Follow Math Help Forum on Facebook and Google+

  8. #8
    Senior Member
    Joined
    May 2010
    From
    Los Angeles, California
    Posts
    274
    Thanks
    1
    Quote Originally Posted by abn84 View Post
    In automation, initial conditions of signals and their different derivatives are often supposed null.
    Important piece of information not obvious to people not in the field!

    You can view the transformation that I did is to apply a double integrator filter to all inputs and outputs:
    Y(s)= H_1(s)*U(s)+H_2(s)*Z(s)   so   F(S)*Y(s)=F(s)*H_1(s)*U(s)+F(s)*H_2(s)*U(s)
    where F(s)= 1/s^2
    Looks like you're using Fourier transforms to solve the d.e. but I'm still confused. You said early that the functions u, y are known but z isn't. But then this d.e. is just a simple first-order d.e. in z. Secondly, you seem to have too many parameters to solve this problem uniquely. I think there must be more info you're not telling us.
    Follow Math Help Forum on Facebook and Google+

  9. #9
    Newbie
    Joined
    Mar 2011
    Posts
    11
    Hi,
    Quote Originally Posted by ojones View Post
    Important piece of information not obvious to people not in the field!
    [...]
    I think there must be more info you're not telling us.
    Sorry, it's not evident to know what information are crutial and what are not. You can ask me what you want.


    Looks like you're using Fourier transforms to solve the d.e. but I'm still confused.
    No, I'm just more familar to write in laplace domain. What I'd like to say is that it is well known, in identification problem, that if you apply the same filter to input and output, the identified system would be strictly the same. In my problem,as I have derivatives, I applied a double itergator to my signals to avoid digital derivation noise.
    You said early that the functions u, y are known but z isn't. But then this d.e. is just a simple first-order d.e. in z.
    Yes, you'r right, however I don't know the parameters of the differential equation.

    Secondly, you seem to have too many parameters to solve this problem uniquely.
    Well, I would reformulate my question more precisely, with examples, it will be better
    Follow Math Help Forum on Facebook and Google+

  10. #10
    Newbie
    Joined
    Mar 2011
    Posts
    11
    The bearbox is described by a differential equation:
    v''=p_1 c'' + p_2 c' - \frac{p_1}{q} v' - \frac{p_2}{q} v - \frac{p_1}{q} d' - \frac{p_2}{q} d
    v and c are measured signals
    p_1, p_2 and q are unknown time varying parameters that I'd like to estimate in real time.
    and d is unmeasured disturbance.
    I did a simulation.
    here you can view c and v:

    the initial value of the parameters is known
    p_1 switches at 5.5s from 35 to 50
    p_2 switches at 9s from 100 to 135
    q switches at 12s from 5 to 8
    d switches at 16s from 0 to -4
    To do real time estimation, I'm performing regression on sliding window of the size of 150 elements (sampled at 2ms).
    As I wrote:
    b= A x
    The regression estimates: x=[p_1, p_2, p1/q, p2/q, d*p1_/q, d*p2_/q]
    Here you can view what I got with ordinary least squares regression

    you can view that the steady estimation of p_1 (blue color) is quite good until q changes. it's transient is however not satisfying.
    the estimation of p_2 is less stable.
    and finally the variation of q diverges all the estimation.

    I tried Tichonov regulation with Q and P chosen by trial/error:

    this emrpved transient attenuation but not the stability of th algorithm.
    My ultimate goal is to estimate p_1, p_2, and q. I think that I'm on the right way to do it but there are some problem on illness of the problem.
    in fact if I run the same algorithm with signals that are persistant, I got the right parameter and even an estimation of d.
    A non-persistent excitation means that the input signal does not fully excite the system. For example, if the system is a linear filter bank with adjustable weights at various frequencies, then the input is non-persistent if it has no spectral content at one or more of these frequencies.
    however as my signals i A and b are derivatives and integrals of each others, I think that this causes the non mersistance and consequenty the illness of the problem. that's why I talked about ill posed problem in the begining of the topic.
    according to you is there a way to improve the estimation of the parameters? and how?
    I hope my explaination is clearer, don't hesitate to ask me further question if needed
    Follow Math Help Forum on Facebook and Google+

  11. #11
    Senior Member
    Joined
    May 2010
    From
    Los Angeles, California
    Posts
    274
    Thanks
    1
    You started this thread by given a problem in terms of integration wrt to t but now you're using Laplace transforms. Your approach is not very consistent. Secondly, I still don't see how a single d.e. determines the parameters p_1, p_2, q and d (I'm assuming that the functions v, c, and d are not vector-valued).

    I'm not convinced you've formulated the problem correctly.
    Follow Math Help Forum on Facebook and Google+

  12. #12
    Newbie
    Joined
    Mar 2011
    Posts
    11
    Hi Jones,
    Thank you for your contributions;
    I will try to answer your questions:
    You started this thread by given a problem in terms of integration wrt to t but now you're using Laplace transforms.
    As I said previously, I'm not using Laplace transform at all. Laplace domain is just convinient to represent filters.
    You can view s just like a differential operator so that s u is the derivative of u, \frac{1}{s} u it's integral and \frac{1}{s^2} u it's second integral.
    It's just a way to write the equation, I'm not using laplace Transform.
    aren't you agree that
    v''=p_1 c'' + p_2 c' - \frac{p_1}{q} v' - \frac{p_2}{q} v - \frac{p_1}{q} d' - \frac{p_2}{q} d
    is strictly equivalent to
    v = p_1 c + p_2 \int_{} c\, \mathrm dt - \frac{p_1}{q} \int_{} v\, \mathrm dt - \frac{p_2}{q} \int_{} \int_{} v\, \mathrm dt^2 - \frac{p_1}{q} \int_{} d\, \mathrm dt - \frac{p_2}{q} \int_{} \int_{} d\, \mathrm dt^2
    as integration keeps linearity?
    What I did here is just to apply a double integrator filter to inputs and outputs.
    I'm may be wrong to integrate the equation?
    anyway, we can write:
    v''=[c'',c',-v',-v,-1',-1]*[p_1,p_2,\frac{p_1}{q}, \frac{p_2}{q},\frac{p_1}{q} d, \frac{p_2}{q} d]
    b= A x
    where b is 1 by n vector, A is 6 by n matrix and x is 6 by one vector
    isn't so?
    couldn't we integrate this linear equation?
    v=[c,\int_{} c\, \mathrm dt,- \int_{} v\, \mathrm dt,- \int_{} \int_{} v\, \mathrm dt^2 ,- \int_{} 1\, \mathrm dt,- \int_{} \int_{} 1\, \mathrm dt^2 ]*[p_1,p_2,\frac{p_1}{q}, \frac{p_2}{q},\frac{p_1}{q} d, \frac{p_2}{q} d]
    In numerical computing, it is not convinient to derivate signals, we generally prefer integration or at least filtering. I however ask my self: may be integration causes a loss of information in the signal? is it? or not?
    If we suppose that the elements of A are uncorrelated, least squares estimation of x is a stable and good one.
    In 1822, Gauss was able to state that the least-squares approach to regression analysis is optimal in the sense that in a linear model where the errors have a mean of zero, are uncorrelated, and have equal variances, the best linear unbiased estimator of the coefficients is the least-squares estimator.
    in this figure you can view the estimation of x for random A matrix:

    You can clearly view that least squares estimates correctly x
    The problem is that, in my problem, signals is A are correlated:
    fist, lines are time series signals.
    secondly, colums are integrals and derivatives of each others.
    So the condition of uncorrelation is not respected.
    Isn't that what we call "ill posed problem"?
    So I'm lookin for a method to correctly estimate x althougth the correlation in signals.

    (I'm assuming that the functions and are not vector-valued).
    no, They are. As, I said previously, sorry I'm perhaps not very clear in my explaination, I apologise, currently I'm using a sliding window of 150 terms so v, c ans d are 1 by 150 vectors (150 is arbitrary, it would be biger or smaller).
    I still don't see how a single d.e. determines the parameters and
    In fact, I have 150 d.e. and 4 unknown parameters, so it would be possible to solve the system in the meaning of least squares.
    There is may be an other better estimator than least squares one? as x=[p_1,p_2,\frac{p_1}{q}, \frac{p_2}{q},\frac{p_1}{q} d, \frac{p_2}{q} d], I should later estimate p_1, p_2 and q from x.
    I'm open to other solutions, if you think, there will be a better way to estimate p_1, p_2 and q than with least square regression.

    Thanks

    Note: may be not obvious:
    when I talk about sliding window, it means that at every instant i (i is time index) I'm conserned with v(i-150:i), c(i-150:i) and d(i-150:i).
    Last edited by abn84; April 1st 2011 at 05:47 AM.
    Follow Math Help Forum on Facebook and Google+

  13. #13
    Senior Member
    Joined
    May 2010
    From
    Los Angeles, California
    Posts
    274
    Thanks
    1
    Sorry but I'm having a lot of trouble understanding you. I would like to help you but the forum is probably not the best place. Send me a private message with a contact email and maybe we can figure this out.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Brainteaser posed by colleague at work
    Posted in the Advanced Statistics Forum
    Replies: 4
    Last Post: April 21st 2011, 11:07 AM
  2. Well-posed Schrödinger equation
    Posted in the Differential Equations Forum
    Replies: 1
    Last Post: March 1st 2011, 09:59 AM
  3. is this a well posed problem ????
    Posted in the Calculus Forum
    Replies: 4
    Last Post: January 11th 2009, 06:56 PM

Search Tags


/mathhelpforum @mathhelpforum