how to regularize an ill posed problem

**abn84** · March 26th 2011, 04:13 PM

Hi,
This is my first contribution to this forum
I have some problems to get regression to this equation:
$u + a_1 \int_{} u\, \mathrm dt + a_2 \int_{} \int_{} u\, \mathrm dt^2 + b_1 \int_{} 1\, \mathrm dt + b_2 \int_{} \int_{} 1\, \mathrm dt^2 - c_1 y - c_2 \int_{} y\, \mathrm dt = 0$
my goal is to estimate the different a,b ans c knowing u and y.
What I know are u and y vectors over time. It would not be difficult to compute the different integrals.

My fist trial was to wright the linear system:
$b = A x$
$u = [-\int_{} u\, \mathrm dt ,-\int_{} \int_{} u\, \mathrm dt^2,- \int_{} 1\, \mathrm dt ,- \int_{} \int_{} 1\, \mathrm dt^2 , y , \int_{} y\, \mathrm dt ] [a_1,a_2,b_1,b_2,c_1,c_2]'$

Like that, the problem seems to be ill posed, in fact the estimation of a, b and c do not converge (the algorithm is unstable). I tried Tikhonov regularization, improved the stability for some parameters but not for all:
$x = x_0 + (A' P A + Q)-^1 A' P (b-A x_0)$
I choosed Q and P ad-hoc.
I think the illness is due to:
- the choice of A and b: why should I wright u= and not $\int_{} u\, \mathrm dt$ = or y= or somethink else?
- The fact that in A there are some signals that are not independent:some signals are the integrals of others.

How can I regularize this ill problem? or how to wright it as a well posed problem? How to consider the integral relationship? rigorously.

thanks

**ojones** · March 29th 2011, 11:39 PM

You haven't indicated the regions over which the integrals are to be done. Do you have a reference for this problem?

**abn84** · March 30th 2011, 12:10 AM

Thank you for answer.

You haven't indicated the regions over which the integrals are to be done.

Sorry, you'r right. it's an integration over time so it's between zero and final instant (t)

Do you have a reference for this problem?

what do you meen by reference?
thanks

**ojones** · March 30th 2011, 12:47 AM

The statement is still confusing. In one instance you're performing a single integral of u but in another you're performing a double integral. How are you viewing u both as a function of one and two variables? You wrote $b=Ax$ but neither $b, A$ or $x$ is defined.

By a reference I mean where did the problem come from. Is it something out of a book or is it something you came up with?

**abn84** · March 30th 2011, 03:10 AM

In one instance you're performing a single integral of u but in another you're performing a double integral. How are you viewing u both as a function of one and two variables?

This is simply a second order differential equation, like for example position is the integral of velocity and the second integral of acceleration. both simple and double integral are between 0 and t.

By a reference I mean where did the problem come from. Is it something out of a book or is it something you came up with?

This equation cames from a real physical model.
Rigorously, the equation of the model is: $u''+a_1 u'+a_2 u+d_1 z'+d_2 z = c_1 y''+c_2 y'$
Physically, derivation is not causal ans causes problem is numerical computing so I integrate que equation twice and obtained:

$u + a_1 \int_0^t u\, \mathrm dt + a_2 \int_0^t \int_0^t u\, \mathrm dt^2 - c_1 y - c_2 \int_0^t y\, \mathrm dt + d_1 \int_0^t z\, \mathrm dt + d_2 \int_0^t \int_0^t z\, \mathrm dt^2 = 0$ .
I know u and y but not z, I however know $z$ is quite constant over time so I wrote:
$d_1 \int_0^t z\, \mathrm dt + d_2 \int_0^t \int_0^t z\, \mathrm dt^2=b_1 \int_0^t 1\, \mathrm dt + b_2 \int_0^t \int_0^t 1\ dt^2$

You wrote but neither or is defined.

My goal is to have a good estimation of $a_1, a_2,c_1$ and $c_2$ .
$b_1$ and $b_2$ or $d_1$ and $d_2$ are not important for me, however as a regressor estimates all x vector, I think that bias on b would causes bias on other values. so a posed arbitrary a linear equation:
$b= A x$
where
$b=u$ ; $A=[-\int_0^t u\, \mathrm dt ,-\int_0^t \int_0^t u\, \mathrm dt^2,- \int_0^t 1\, \mathrm dt ,- \int_0^t \int_0^t 1\, \mathrm dt^2 , y , \int_0^t y\, \mathrm dt ]$
and computed x by least mean square regressor witch gives me an estimation of:
$x=[a_1,a_2,b_1,b_2,c_1,c_2]'$
-------------------------------------------
an other information that I don't know how to exploit is the fact that:
$a_1/a_2=c_1/c_2=d_1/d_2$

I hope this contribution is little bit clearer than previously.

Thanks for contribution

**ojones** · March 30th 2011, 11:08 AM

OK, so the original problem was a differential equation, not an integral equation. I can see immediately that you are missing initial conditions on the unknown functions.

You should probably describe the physical situation that gave rise to this formulation. It doesn't look like you've done it correctly because, in addition to not specifying initial coinditions, you seem to have too many unknown functions and parameters.

**abn84** · March 30th 2011, 02:53 PM

Hi Jones,
Previous works (not mine) in our company gave this differential equation of a special type of gearbox where
u is a measured input torque
y is a measured velocity
ans z is an unmeasured disturbance torque
It doesn't matter if I don't know the disturbance but I need to know correctly th parameters.

I can see immediately that you are missing initial conditions on the unknown functions.

In automation, initial conditions of signals and their different derivatives are often supposed null.
You can view the transformation that I did is to apply a double integrator filter to all inputs and outputs:
$Y(s)= H_1(s)*U(s)+H_2(s)*Z(s) so F(S)*Y(s)=F(s)*H_1(s)*U(s)+F(s)*H_2(s)*U(s)$
where $F(s)= 1/s^2$
I will give you tomorrow a set of data and resultsview better the problem.

Thanks

**ojones** · March 30th 2011, 03:45 PM

Originally Posted by abn84

In automation, initial conditions of signals and their different derivatives are often supposed null.

Important piece of information not obvious to people not in the field!

You can view the transformation that I did is to apply a double integrator filter to all inputs and outputs:
$Y(s)= H_1(s)*U(s)+H_2(s)*Z(s) so F(S)*Y(s)=F(s)*H_1(s)*U(s)+F(s)*H_2(s)*U(s)$
where $F(s)= 1/s^2$

Looks like you're using Fourier transforms to solve the d.e. but I'm still confused. You said early that the functions $u, y$ are known but $z$ isn't. But then this d.e. is just a simple first-order d.e. in $z$ . Secondly, you seem to have too many parameters to solve this problem uniquely. I think there must be more info you're not telling us.

**abn84** · March 31st 2011, 01:25 PM

Hi,

Originally Posted by ojones

Important piece of information not obvious to people not in the field!
[...]
I think there must be more info you're not telling us.

Sorry, it's not evident to know what information are crutial and what are not. You can ask me what you want.

Looks like you're using Fourier transforms to solve the d.e. but I'm still confused.

No, I'm just more familar to write in laplace domain. What I'd like to say is that it is well known, in identification problem, that if you apply the same filter to input and output, the identified system would be strictly the same. In my problem,as I have derivatives, I applied a double itergator to my signals to avoid digital derivation noise.

You said early that the functions $u, y$ are known but $z$ isn't. But then this d.e. is just a simple first-order d.e. in $z$ .

Yes, you'r right, however I don't know the parameters of the differential equation.

Secondly, you seem to have too many parameters to solve this problem uniquely.

Well, I would reformulate my question more precisely, with examples, it will be better

**abn84** · March 31st 2011, 03:16 PM

The bearbox is described by a differential equation:
$v''=p_1 c'' + p_2 c' - \frac{p_1}{q} v' - \frac{p_2}{q} v - \frac{p_1}{q} d' - \frac{p_2}{q} d$
$v$ and $c$ are measured signals
$p_1$ , p_2 and q are unknown time varying parameters that I'd like to estimate in real time.
and d is unmeasured disturbance.
I did a simulation.
here you can view c and v:

the initial value of the parameters is known
$p_1$ switches at 5.5s from 35 to 50
$p_2$ switches at 9s from 100 to 135
$q$ switches at 12s from 5 to 8
$d$ switches at 16s from 0 to -4
To do real time estimation, I'm performing regression on sliding window of the size of 150 elements (sampled at 2ms).
As I wrote:
$b= A x$
The regression estimates: $x=[p_1, p_2, p1/q, p2/q, d*p1_/q, d*p2_/q]$
Here you can view what I got with ordinary least squares regression

you can view that the steady estimation of $p_1$ (blue color) is quite good until $q$ changes. it's transient is however not satisfying.
the estimation of $p_2$ is less stable.
and finally the variation of $q$ diverges all the estimation.

I tried Tichonov regulation with Q and P chosen by trial/error:

this emrpved transient attenuation but not the stability of th algorithm.
My ultimate goal is to estimate $p_1$ , $p_2$ , and $q$ . I think that I'm on the right way to do it but there are some problem on illness of the problem.
in fact if I run the same algorithm with signals that are persistant, I got the right parameter and even an estimation of d.

A non-persistent excitation means that the input signal does not fully excite the system. For example, if the system is a linear filter bank with adjustable weights at various frequencies, then the input is non-persistent if it has no spectral content at one or more of these frequencies.

however as my signals i A and b are derivatives and integrals of each others, I think that this causes the non mersistance and consequenty the illness of the problem. that's why I talked about ill posed problem in the begining of the topic.
according to you is there a way to improve the estimation of the parameters? and how?
I hope my explaination is clearer, don't hesitate to ask me further question if needed

**ojones** · March 31st 2011, 03:50 PM

You started this thread by given a problem in terms of integration wrt to t but now you're using Laplace transforms. Your approach is not very consistent. Secondly, I still don't see how a single d.e. determines the parameters $p_1, p_2, q$ and $d$ (I'm assuming that the functions $v, c,$ and $d$ are not vector-valued).

I'm not convinced you've formulated the problem correctly.

**abn84** · April 1st 2011, 04:19 AM

Hi Jones,
Thank you for your contributions;
I will try to answer your questions:

You started this thread by given a problem in terms of integration wrt to t but now you're using Laplace transforms.

As I said previously, I'm not using Laplace transform at all. Laplace domain is just convinient to represent filters.
You can view $s$ just like a differential operator so that $s u$ is the derivative of $u$ , $\frac{1}{s} u$ it's integral and $\frac{1}{s^2} u$ it's second integral.
It's just a way to write the equation, I'm not using laplace Transform.
aren't you agree that
$v''=p_1 c'' + p_2 c' - \frac{p_1}{q} v' - \frac{p_2}{q} v - \frac{p_1}{q} d' - \frac{p_2}{q} d$
is strictly equivalent to
$v = p_1 c + p_2 \int_{} c\, \mathrm dt - \frac{p_1}{q} \int_{} v\, \mathrm dt - \frac{p_2}{q} \int_{} \int_{} v\, \mathrm dt^2 - \frac{p_1}{q} \int_{} d\, \mathrm dt - \frac{p_2}{q} \int_{} \int_{} d\, \mathrm dt^2$
as integration keeps linearity?
What I did here is just to apply a double integrator filter to inputs and outputs.
I'm may be wrong to integrate the equation?
anyway, we can write:
$v''=[c'',c',-v',-v,-1',-1]*[p_1,p_2,\frac{p_1}{q}, \frac{p_2}{q},\frac{p_1}{q} d, \frac{p_2}{q} d]$
$b= A x$
where $b$ is 1 by n vector, $A$ is 6 by n matrix and $x$ is 6 by one vector
isn't so?
couldn't we integrate this linear equation?
$v=[c,\int_{} c\, \mathrm dt,- \int_{} v\, \mathrm dt,- \int_{} \int_{} v\, \mathrm dt^2 ,- \int_{} 1\, \mathrm dt,- \int_{} \int_{} 1\, \mathrm dt^2 ]*[p_1,p_2,\frac{p_1}{q}, \frac{p_2}{q},\frac{p_1}{q} d, \frac{p_2}{q} d]$
In numerical computing, it is not convinient to derivate signals, we generally prefer integration or at least filtering. I however ask my self: may be integration causes a loss of information in the signal? is it? or not?
If we suppose that the elements of A are uncorrelated, least squares estimation of x is a stable and good one.

In 1822, Gauss was able to state that the least-squares approach to regression analysis is optimal in the sense that in a linear model where the errors have a mean of zero, are uncorrelated, and have equal variances, the best linear unbiased estimator of the coefficients is the least-squares estimator.

in this figure you can view the estimation of x for random A matrix:

You can clearly view that least squares estimates correctly $x$
The problem is that, in my problem, signals is A are correlated:
fist, lines are time series signals.
secondly, colums are integrals and derivatives of each others.
So the condition of uncorrelation is not respected.
Isn't that what we call "ill posed problem"?
So I'm lookin for a method to correctly estimate $x$ althougth the correlation in signals.

(I'm assuming that the functions and are not vector-valued).

no, They are. As, I said previously, sorry I'm perhaps not very clear in my explaination, I apologise, currently I'm using a sliding window of 150 terms so $v$ , $c$ ans $d$ are 1 by 150 vectors (150 is arbitrary, it would be biger or smaller).

I still don't see how a single d.e. determines the parameters and

In fact, I have 150 d.e. and 4 unknown parameters, so it would be possible to solve the system in the meaning of least squares.
There is may be an other better estimator than least squares one? as $x=[p_1,p_2,\frac{p_1}{q}, \frac{p_2}{q},\frac{p_1}{q} d, \frac{p_2}{q} d]$ , I should later estimate $p_1$ , $p_2$ and $q$ from $x$ .
I'm open to other solutions, if you think, there will be a better way to estimate $p_1$ , $p_2$ and $q$ than with least square regression.

Thanks

Note: may be not obvious:
when I talk about sliding window, it means that at every instant i (i is time index) I'm conserned with $v(i-150:i)$ , $c(i-150:i)$ and $d(i-150:i)$ .

**ojones** · April 1st 2011, 03:24 PM

Sorry but I'm having a lot of trouble understanding you. I would like to help you but the forum is probably not the best place. Send me a private message with a contact email and maybe we can figure this out.

Thread: how to regularize an ill posed problem

LinkBack

Thread Tools

Display

how to regularize an ill posed problem

Similar Math Help Forum Discussions

Brainteaser posed by colleague at work

Well-posed Schrödinger equation

is this a well posed problem ????

Search Tags