The Fundamental Theorem of Calculus or Why You Hate Calculus Part II Or Why Are Integration and Differentiation Inverse Operations?

Differentiation finds the gradient of a curve. Integration finds the area between a curve and the horizontal axis. Not self-evidently inverse/reverse/converse procedures. Well, not upon the same function at least. The basic trick is to think of differentiation and integration as operations which turn one function into another.

Recall that a function f is the totality of all (x, y) ordered pairs, where x is a number in the domain of the function f and y is that number in the range of the function f associated with x. When graphing a function we plot those number pairs on a Cartesian plane ( mind you, there are functions which have many variables in the domain, and many in the range. Letâ€™s not talk of those .... ) We thus can write y = f(x) to indicate that f â€˜transportsâ€™ x to y. With certain restrictions one can define an inverse function, call it f^[-1], which is based uponfâ€™s behaviour so that x = f^[-1](y) is a sensible statement : meaning that f^[-1] â€˜transportsâ€™ some y back to the x from whence it first came via the action of f. When this is allowed it is a symmetric situation though, so which function has the superscipt/exponent of -1, reminding us of the numerical idea of â€˜reciprocalâ€™, is a matter of taste. I could give f's inverse the name g for instance, and simply say that f and g are each other's inverses. This you would write as f = g^[-1] and g = f^[-1]. Especially note that f^[-1]does not denote the function which forms the algebraic multiplicative inverse that we call division, although conceivably the manner in which the functions convert xy values might involve that.

So if you have some set/collection of functions then it could be the situation where applying the ( limit based ) processes involved in integration and differentiation might allow you to transfer between one function and another function. Thus it can be the case that differentiating a function h produces the function f, and thus integrating/anti-differentiatingf produces the function h. f is therefore called the derivative of h and is thus the function which represents the tangent gradients for each applicable point ( to be discussed ... ) in the domain of h. h will be called the anti-derivative or integral of f. We will show that h ( potentially at least ) represents either a cumulative area sum up to a given point in the domain of f or simply 'that function which when differentiated produces f'. I guess it's an historical thingy, but no one talks of the terms 'anti-integration', 'anti-integral' etc. Whew!! We could show these concepts like so :

where the set has functions as it's members, and thus is not either the domain set or range set of numbers upon which functions are defined to act upon. Again, differentiation and integration are operations upon functions - which is a rather higher level concept than a function associating members of a domain set to those within a range set.

Now weâ€™ve discussed integration which has the following graphical/geometric interpretation :

this is a definite integral, as discussed, with particular choices of the endpoints A and B. Differentiation, which we didnâ€™t discuss, looks like this ( with various annotations for the derivative ) :

Maybe thus it is time to explain differentiation, otherwise this topic will make little sense. I should have perhaps brought it up earlier. I thought we could get away with it, but the square-wave/box-function analysis just done required at least a basic understanding of anti-differentiation .....

The line that typifies the â€˜slope of the curveâ€™ is known as the tangent line. The tangent â€˜just touchesâ€™ the curve at one point. Gradient is generically phrased as being â€˜rise divided by runâ€™ or a ratio of two lengths, the rise being how far up/down one goes and the run being how far along/back. There will be a problem if the run is zero, as that means 'divide by zero' and we will never be doing that in what follows. A rise of zero is OK though. You can satisfactorily analogise/visualise here by thoughts of roads, hillsides, roofing styles and aeroplane flightpaths etc .... if you like. The rise, the run and hence the gradient are signed quantities. By convention - positive is for rise going up and run going to the right, negative is for rise going down and run going to the left. That is, the same sense as the standard Cartesian orientation of x and y axis directions. Differentiation is a limiting process, where we approximate some quantity by successively finer distinctions. To this end I introduce the secant line which is going to be our approximating mechanism :

and youâ€™ll note that the secant intersects our curve at two points, whereas the tangent does so at only one point. The secant line and the tangent line have one point in common â€“ the point at which we wish to evaluate the slope/gradient/derivative. This brings up a very important point : whereas integration involved an entire ( and possibly quite broad, or even the whole ) subset of fâ€™s domain and range, differentiation is concerned almost exclusively with progressively smaller chunks of fâ€™s domain and range. In the strictest sense the Q's only need to be arbitrarily close to P, not arbitrarily far away from P. So you could term differentiation as a local procedure with integration being of a global nature. Although I've drawn Q on the right side of P, we need to allow Q to come from the left as well. Regardless of which side Q comes from the secant gradient limit must be the same. Note that while both the quantities |f(b) - f(a)| and |b - a| go towards zero the ratio of the two may not - indeed that is the key point in finding a 'rate of change'.

As with integration you can work from the basic definitions : pick a function, pick a point, form the algebraic quantity that is the secant gradient, express that in terms of the distance |b - a| and study what limit value that secant gradient approaches as |b - a| goes toward zero ( but is never zero ) - the whole epsilon thingy. OR you can use general theorems which have been deduced from that base level and work with those - because life is too short! :-)

Now the limit of the secant gradient can't always be reached. The main cause of failure is discontinuity of the function, which generally means some 'gap' in the curve and there are no P's or Q's to speak of in the gap that we can examine. But there are other reasons like the function not being defined at all on some interval, or the 'pointy-ness' mentioned earlier that causes the left sided gradient limit to be not equal to the right sided gradient limit. The endpoints of intervals can be an issue too, say where you have a function defined on a closed interval - one that includes the endpoints in the set - like [A, B] but not outside that ie. if x B. We use square brackets '[' or ']' when specifying an interval to indicate if an endpoint is included, and round ones '(' or ')' if an endpoint isn't. Then if you want the derivative at A you can't form any left sided limits as the function has no values there ie. there is no Q ( you might get a derivative at any particular point just to the right of A as there will be some points between such a chosen point and A to work with ). Likewise for any right sided limits for B. Thus the function may have it's derivative defined for the interior of the interval, meaning at every point except the endpoints ie. (A, B). Whew!! :-)

Now take some function for which I can form the integral as follows :

h(t) represents the area under the curve from x = 0 to x = t. Now as f(0) = 0 then h(0) = 0 too, as there is no initial area under the curve. So h(2) say would be the area, as calculated by the integral, b/w the curve f(x) and the x-axis bounded by x = 0 and x = 2. Just imagine sliding the x = t point on the x-axis back and forth and thinking how h(t) will vary as you do that. As the curve is drawn then h(t) will increase as t goes to the right - as more area is covered by [0, t] - and h(t) will decrease as t goes back to the left.

Here is the key point : the greater f(x) is at some point the more area b/w it and the axis will be added into the integral that defines the function we have called h(t). We showed this earlier as those tall thin boxes contributing to the integral sums via the Reimann method. Hence how quickly h(t) changes is proportional to f(x), as if f(x) is low then only a little area adds in whereas if f(x) is high alot of area is added - for equal ( small ) steps along the x direction. If f(x) suddenly went to zero at some point then no area is involved and h(t) would not increase and would reflect the cumulative area under the curve up until the point when f(x) did go to zero.

Algebraically if dt is the little extra bit of increase in t then :

the increase in area = h(t + dt) - h(t) ~ f(t) * dt

thus f is about equal to the rise/run on the curve of h(t). If you then take the limit of the rise/run ratio as dt goes to zero then f becomes exactly equal to the rate of change of h. Hence the function f is the derivative of the function h, where h is the function representing f's integral ...... TA DA !!! :-) :-)

Note, yet again, that The Fundamental Theorem Of Calculus does not speak of the derivative and the integral of a single function - it says the derivative of one function h is another function f, and the integral of f is thus h.

Go back to the supermarket queue if that helps. How quickly the running total of your grocery bill increases, item by item, depends on the price of the items as they are scanned. Add in a cheap item then the total increases a small amount, add in an expensive one then the total jumps by a fair bit when that item is scanned through .... :-)

Cheers, Mike.

( edit ) The reason I first discussed functions and their inverses was to (a) emphasise such detail about that, as was not previously mentioned but I reckon we'll need and (b) to have something to contrast against - functions operating on numbers as opposed to operations on functions.

( edit ) Try this : a sequence of N ( = 10 ) numbers, a(n) = {1, 5, 7, 2, 0, 6, 3, 1, 1, 5} where n = 1 ...... 10

So the basic reason why integration and differentiation are opposite operations on functions is the same as why addition and subtraction are opposite operations on numbers. With our sequence & series example here we used integers to index the values, with integration/differentiation we use real numbers to index the variables.

I have made this letter longer than usual because I lack the time to make it shorter.Blaise Pascal

Okey dokey, another Fourier co-efficients example, a triangle wave :

which upon suitable analysis has the following line of C(k) lollipops :

which are all real, non-negative and more savagely diminishing as k increases than was the case with the box function. C(0) = 1/2 too. We can likewise animate accumulating contributions to the waveform from these co-efficients :

which are cycling from k = 0 to 21 and back down again to k = 0. You get the triangle wave forming real quick. These partial sums are of the form :

given that the 'total size' of the function is about order 1, then a number like (4/81*PI^2) ~ 0.0012 ~ 0.1% is quite a small correction ( cosine of anything has magnitude no greater than 1 ) and occurring only in the first handful of terms. The triangle being continuous ( though not everywhere differentiable ) whereas the box is not continuous everywhere on it's quoted domain. BTW if a function fails to be continuous and/or differentiable even at only a single point on it's domain, it is awarded the relevant title 'discontinuous' or 'non-differentiable' anyway - a strict club indeed !

Box Function Redux I didn't mention the partial sums for you to look at and compare with the triangle function's :

Unfortunately I drew the box cliff/sides as an unbroken vertical line, I really should have graphed as follows :

to emphasis the gap, for at t = +1/2 or -1/2 you can only have one point above or below. You can choose the cliff top, bottom or half way up ( or whatever ) say for f(1/2) and f(-1/2) - but not the whole cliff face at once! Otherwise we abrogate the definition of a function, meaning you can't have a plurality of values in the range for a single value in the domain. Sorry for my bad in the graphing, though it wasn't relied upon in the subsequent calculations.

However this does raise a rather interesting point. If one takes the C(k)'s as analysed for the box function and plug them back into the synthesis equation :

and then use that to evaluate f(1/2) and f(-1/2), one gets the answer f(1/2) = f(-1/2) = 1/2 regardless of the original choice of f(1/2) or f(-1/2) in the definition! So the initial placement of those two points is really irrelevant amongst the continuous infinity of points in the time domain, in fact you could even refuse to define the function at all with those particular two points ie. exclude them from the domain. The process - from function prescription to analysis of co-efficients to synthesis leading back to the function again - somehow 'knows' what the average value of the function at t = -1/2 or 1/2 'should' be : halfway up the cliff. Apparently this applies for other similiar 'gap' examples, and there is an awfully deep reason why this should be so ( which I haven't uncovered yet ) but my guess is related to integrals being a global process on a function ........ ;-)

[ Now this segway's into the another idea which is more relevant to what happens at E@H : what if we don't have a continuous record of f(t) over all the time domain? Mathematically we might hypothecate a continuous infinity of points in a finite range ( eg. the real number interval [0,1] ), but in reality that's not going to be what's measured. Firstly we can't store an infinite number of values and secondly we can't 'slice' time into infinitely narrow intervals. What we will get is a sampling of a function : the photodetectors will average out a lot of photons received within some short time period AND while that is an 'analogue' signal ( to the extent that quantum mechanics truly allows ) we will still be converting that to a digital representation with a granularity determined by the chosen bit length. In short our fancy integrals don't fly. What is then used goes generally by the moniker of the Discrete Fourier Transform ( though some say is more accurately described by the phrase 'Discrete Fourier Series' ), these days enacted/implemented by the very efficient Fast Fourier Transform algorithm. But I'll get to that after some more preparation on my side ... :-) ]

Now earlier we looked at Rayleigh's Identity :

I won't bore you with the proof detail, the key step of which is interchanging integration with summation, but discuss further. Physically ( say for a light signal composed of photons ) it says that within one cycle of a periodic wave the power ( energy received per that time interval ) is the same whether you add it up in the time domain or the frequency domain. Let's illustrate/imagine with a thought experiment :

You are in a dark room with a photodetector measuring some periodic signal delivered to you in the form of a beam of light coming from an adjacent room through a suitable hole in one wall. I am in that other room which a bunch of apparatus, lasers etc. You can analyse the signal into it's Fourier co-efficients, presumably to a precision/extent limited by our mutual exhaustion and other practical factors - no matter. Now you wouldn't know it, but I'm in that adjacent room with a bunch of very special lasers, mirrors and whatnot - as many as I can afford and/or fit in - and that each laser is producing a 'pure tone' related to the fundamental frequency of the periodic signal you are studying ie. base plus harmonics. I can modulate the power and the relative phases of each of those lasers and shine the summation ( light combines linearly ) through the wall hole to reach your photodetector. Your calculated Fourier co-efficients will directly relate, with only questions of scaling and translation, to my laser and apparatus settings! Here's what Rayleigh's Identity means : you can count the photons moment to moment and sum to get the LHS, whereas I can add up those same photons but laser by laser and get the RHS!! The total photon energies involved going through the hole b/w the two rooms is simply accounted for in a different manner either side of the wall. This goes back to the vector space analogy too - are you looking at the variation in some total vector length ( LHS of Rayleigh's Identity using photodetectors ) OR or am I following that vector's component values ( RHS of Rayleigh's Identity using laser settings ) ?? As the RHS is a sum of squares of component 'lengths', albeit infinite in number, then Rayleigh's Identity is really Pythagorus' Theorem on infinite dimensional steroids! :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter.Blaise Pascal

Box Function Redux Redux I should have been more explicit on the f(1/2) story. If t = 1/2 then cos(PI/2) = cos(3*PI/2) = cos(5*PI/2) = ..... = 0 and same for t = -1/2, that is odd multiples of PI/2 is where the cosine function crosses the horizontal axis. That makes all the terms in the Fourier series zero, except the first which is the constant term of value 1/2. I guess the reason isn't so mysterious ( though the exact proof might be ) - as the constant DC term is the function's average then altering the placement of a few points out of the infinitely many used on the interval of integration ( that derived said constant term ) makes no difference to an average. This rather emphasises the idea that an integral is a 'broad look at' a function over an interval*.

The Story Thus Far With the ultimate aim of understanding signal processing, we started by considering how to break apart, in a linear fashion, a given function into sub-parts. We discarded the power functions as too unruly to use for periodic behaviours. The obvious, but none-the-less quite bright, idea of using periodic functions as sub-units came along and thus the sine and cosine were chosen - which are well understood and have a long pedigree. Being based on that simplest of geometric shapes ( the circle ) you get regularity 'for free' so to speak. These were then combined into the semantically and algebraically tricky - have just a glance at the co-efficient derivations I parked in some links earlier, they certainly made me lollygag a bit in preparation & I never got them right first time - but highly useful new type of number called complex. Try thinking of complex numbers as an abstraction that allows the messy stuff to be left in the engine room while you steer the ship on the bridge. The complexity never leaves ie. no free lunches ( & no pun intended ) but it's managed at a lower level while you get to write and manipulate high level concepts with a concise notation. One should never underestimate the cognitive value of a good notation, we humans seem to need words to express concepts and in math the notations form the 'words'. Then we talked of limits and particular applications of that, specifically integrals which are one way of looking at the totality of some function. Then vectors and the spaces they live in were introduced for the purposes of analogy, with the great concepts of orthonormality, linear independence and projection. All along we kept refining the detail of how to represent functions as series/sums of (co)sinusoids. Then we talked of periodicity in general and the 'one period but many frequencies' rule, finally getting the explicit expression to use when calculating those all important Fourier coefficients with a particular function presented for analysis into sub-components. We did eventually get the rubber to meet the road with some examples, which will also have great utility and relevance later on. We had some snacks & detours along the way like the Fundamental Theorem of Calculus, continuity, differentiation, and another important higher level concept : operations upon functions. Here's some key features to highlight/note again :

- the C(k)'s are the multipliers of the oscillating base functions.

- the DC or k = 0 component doesn't itself oscillate but represents the average level about which others do.

- because we deal with real signals, the +ve and -ve C(k)'s are complex conjugates [ so when you add C(k) times it's oscillating complex exponential to C(-k) times it's oscillating complex exponential then you get a real number ].

- lower frequencies paint the 'broader strokes' while higher frequencies fill in the 'finer detail' like sharp edges/cliffs and pointy bits.

- those pointy bits are correctly represented only as the limit of some progressive approximating process.

- if convergence occurs then there will in general be different 'rates of approach' to the limiting value ie. how many terms to use in the partial series to get within some tolerance level. The rate(s) of change of the function ie. derivative(s) determine this.

Now we ought next wander along to the topic of convolution which is an especial way of combining two functions in one domain that leads to their multiplication in the other domain ( that being reached via some Fourier operation ). The way it will be expressed has a nice symmetry, but to do that we need to expand our Fourier representation of functions to a continuous frequency domain ie. beyond integer multiples and deal with non-periodic functions. It is only much, much later that one will see the true reasons for that route. Plus there will be some nice views along the way ..... :-) :-)

However for the moment I thought we'd roll off the mathematical throttle a bit, coast along, and next show some animations that I'm cooking up - a potpourri of my fiddlings with 3dsMax - to illustrate some of the stuff mentioned so far. And since you're all rather quiet listeners, but I know that several dozen are following along ( some have PM'ed me with further encouragement, so thanks for that ) it could also be a good time to invite questions upon the material shown to date. If you think/worry that you might 'pollute' this thread then feel free to fire up another, not that it would worry me either way. :-) :-)

* which is to say that any of the C(k)'s requires an evaluation of an integral involving the signal function over an entire cycle of the signal. Although I solved for the general k'th case, you can still consider each component frequency requiring it's very own integral specified by plugging in a specific value of k where appropriate in the general Fourier coefficient formula. Indeed if you put k = 0 into that formula, then because exp[zero] = 1, it simplifies to the traditional averaging formula :

here the RHS of that last equality ( in red ) is a bit like finding the length of one side of some rectangle by dividing the rectangle's total area by the width along the other ( orthogonal ) dimension : since area = length * width. You can think of the function's values ( height above the horizontal axis ) as a variable length rectangle side - which is why we added up lots of little rectangles with the Reimann method.

Cheers, Mike.

( edit ) Indeed another way of showing this is : the area b/w the curve and the x-axis could be replaced, on equal area grounds, with a rectangle that has one dimension being the height representing the function's average value and the integration interval being the other dimension :

I have made this letter longer than usual because I lack the time to make it shorter.Blaise Pascal

All this is linear algebra. I passed my exam with professor Giovanni Prodi, the eldest of the 5 Prodi brothers, all university professors, but that was a long time ago. My brother Franco was Prodi's assistant professor at Pisa, where Ennio DeGiorgi and Enrico Bombieri, now at Princeton Institute for Advanced Study were the stars. But that was a long time ago. Now I am cooperating with people from CERN on a BOINC project including a virtual machine, and also at QMC@home on quantum chemistry. Cheers.
Tullio

Ok, time for some animations. I'll post links with the idea of selecting each to open in a new browser tab as you choose/please, rather than force load them all by inline image inclusion -> because the file sizes are up to ~ 4 MB. They are in some perspective view looking upon a segment of the Argand plane. Sort of like looking at a blackboard, as I use the 'third' dimension here.

Rotating complex exponential - say of 'unit' length and frequency 'one' :

..... note the marking of angles as positive for anti-clockwise and negative for clockwise, but any angular value may differ by any multiple of 2*PI when representing the same 'propellor tip' position on the circle.

Now the same but draw the time axis 'out of the blackboard' so the base of the propellor moves along it, and the tip will trace a spiral/helix upon the surface of a cylinder that could be projected out of the board from the unit circle :

.... only two cycles are shown and repeated from t=0 to t=2T, but you could keep going forever in the direction out of the blackboard!

Now for that helix, if you look down along the positive iy-axis ( as if you were sitting on the top rail of the blackboard ) you perceive the horizontal or x-axis component oscillating only - the cosine that is :

and

.... I deliberately didn't rotate the cosine view to match the usual textbook appearance, so as to emphasise the orthogonality of the concept of sine and cosine. Think back to the definition : while the radius is never zero, the cosine measures the x-axis deflection as a signed fraction of the radius and the sine measures the y-axis deflection as a signed fraction of the radius. This also re-asserts the idea that sine and cosine are time/phase shifted versions of the same shape.

Similiarly if you look along the negative x-axis toward the origin ( from the left side of the blackboard ) you perceive the vertical or iy-axis component oscillating only - the sine that is :

and

.... smart punters will note that the cosine looks the same whether we rotate either way, this is what is meant by the cosine being an even function. The sine one way looks like minus the sine the other way, thus sine is deemed as an odd function - because the initial y-axis deflection is upwards for anti-clockwise rotation and downwards for clockwise.

Let's chuck in some

different frequencies now, anti-clockwise rotation, and you'll note that when the blue guy makes two revolutions, the purple guy does one, the green guy does three, and the brown guy four. Time passes the same for all - the tails all proceed at the same rate along the time axis - but the rate of phase/angle change with time varies. This is of course the frequency, as in cycles per second ( or radians per hour, or whatever according to your phase and time units ). Because these frequencies have some integral multiple relationship then they will all come back to knock on the door at some same time, and repeat that co-incidence only whenever the slowest cycling component does so. The

cosine and

sine views aren't as clear ( the propellors overlie one another ) but show the periodicities.

If you're not used to this topic, sines etc then just stare at them for a while. The relationships and the algebra thereof already mentioned will come to view/click. There will be more .... :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter.Blaise Pascal

I want to illustrate the ( partial ) Fourier synthesis of the box function that we've already analysed into Fourier coefficients :

This animation will take some detail to explain fully. Firstly go back to the ideas of adding and multiplying complex numbers :

z_1 = x_1 + iy_1

z_2 = x_2 + iy_2

gives

z_1 + z_2 = (x_1 + x_2) + i(y_1 + y_2)

which if you think of as an ordinary vector on a two dimensional plane, this idea translates to taking the new total vector as being the one where the base of second was placed at the tip of the first thus :

which doesn't have a convenient 'plain vector' analogy, alas. Now this :

is a sum of products. So for each choice of frequency ( k value ) I multiply the relevant complex exponential ( which will be rotating clockwise or anti-clockwise depending on the sign of k, and the rotation speed is proportional to the magnitude of k ) by the corresponding Fourier co-efficient which is also k-dependent. The DC component for the box function is 1/2 and that can be thought of as 1/2 times exp[0], where exp[0] = 1 means length one and phase zero.

The DC is

animated quite simply with the time axis coming out of the blackboard, where the green circle on the Argand Blackboard is no longer a unit circle - but everything will be scaled OK. This also shows the path traced by the propellor tip ( a non-rotating propellor that is! ). This path is aligned in direction with the x-axis which it should because we are modelling a real number valued signal that has no nett imaginary component. For what follows I'll assume the base/fundamental frequency is one ie. T = period = 1. So k/T = k for simplicity.

This brings us to the k = 1 and k = -1 components which we promised/agreed earlier to add in only when together, and so being complex conjugates the nett would remain a real number. For k = 1 then C(1) = 1/PI = 1/PI*exp[0] so the length is a bit less than the DC part ( as PI > 3 ) and the phase offset is zero, and thus it's propellor is :

For k = - 1 it's nearly the same except that exp[PI*i] = -1, and thus this propellor has to begin with a phase offset of PI radians or 180 degrees around the circle, and it rotates the other way too , so we have :

where the t - 1/2 ( for period of 1 ) indicates a lead, or a lag for that matter, of half a cycle. Put

all this together, and here I animate the additions by head-to-tailing the propellors/vectors, so I can look at the path of the tip of the last chained/jointed propellor to follow the total as it evolves over time. You'll note how the contra-rotating propellors conspire by design to keep that tip always upon some direct perpendicular ( to the Argand plane ) line from the x-axis.

The k = +/- 2, +/- 4, +/- 6 .... etc components require no work at all. Herewith includes up to the

k = +3/-3, and then

k = +5/-5 contributions with it's

'sine' or imaginary view ( pretty boring because we are edge on to the real-axis/t-axis plane ) and the

'cosine' or real view ( far more interesting showing some development of the box shape after the few terms I've animated ).

This all rather reminds me of (a) the 'Spirograph' drawing kit/toy I had as a lad (b) a Hewlett Packard plotter my Dad used to output his engineering drawings from a real early version of AutoCAD + SpaceGas ( finite element structural modeller ) (c) epicycles of either Ptolemaic or Copernican construction. Gauss in effect invented his own version of the FFT ( though not calling it that ) when attempting to interpolate trigonometric functions ( for published/printed look up tables ), and when considering asteroid orbits where by interpolate we also mean predict.

Ok, I'll sticky this and start afresh next time with a new thread : onwards and upwards to the continuous frequency domain ..... :-)

Cheers, Mike.

( edit ) I guess it may well be bleedin' obvious to state but : the animations show the progressive tendency toward convergence of the series by the smaller lengths of the oscillating elements, plus you can see how the sum up to a given level acts as the base for further refinement of the curve, also the phases are crucial to making the trend corrections in the right directions, and how the faster oscillating components really do decide upon the finer curve detail as the slow ones just can't move quick enough to do that. All of these issues about lengths/phases/frequencies really do have a correspondence in one's thinking about some real signal analysis problem. Ignoring matters of polarisation, that's all we really receive from distant objects like pulsar systems ... a sequence of time indexed measurements.

I have made this letter longer than usual because I lack the time to make it shorter.Blaise Pascal

... I guess it may well be bleedin' obvious to state but ...

Obvious is good! :-)

The animations very nicely show how the convergence improves as you include ever more resultant points to make the inverse transform increasingly more precise.

I also found it rather interesting to note in what way the transforms return an imprecise version of the original when you restrict the number of transform points. I guess that can be thought of as analogous to undamped filtering the original signal...

Another aside is that the human ear-cochleae-brain system essentially transforms the sound compression waves into a pitch spectrum ('Fourier transform') that we then experience as pitch, tone, relative loudness, and other qualities.

The animations very nicely show how the convergence improves as you include ever more resultant points to make the inverse transform increasingly more precise.

I also found it rather interesting to note in what way the transforms return an imprecise version of the original when you restrict the number of transform points. I guess that can be thought of as analogous to undamped filtering the original signal...

Yup, applying a frequency filter ie. in this case by only going a certain distance along a series from the start ( so this will be a low-pass filter ), has the effect of altering the back-transform ( you could call that convolution ). Indeed that sequence is key to most signal processing tasks - do the transform, manipulate in the alternate domain, then do the back transform to the original domain.

The 'center of the maze' in this whole topic, which I'm aiming at, is where you take some once-off waveform and then periodise ie. repeat/duplicate indefinitely in the time domain ( by applying a 'delta comb' ) and then you restrict that result by applying a single box function to get back to the original non-periodic shape. Although that seems a long way around to come back home, the effect will be that you gain the ability to sensibly interpolate in situations where you have only a discrete number of measurements. So you can deduce what the in-between points 'ought be' in be in circumstances where you can only approximate the curve. But you have to place limits on the rate of variance of the curve between measured points ie. a high frequency cut-off, or you won't sufficiently narrow the field of function candidates that could be used to interpolate values. This I reckon is the jewel in the crown, as this gives the DFT/FFT and hence one can search for buried signals in real data. Gauss used this to answer questions about orbiting bodies in the solar system ie. where were they earlier and where will they be later on? Admittedly there is some messy detail relating to the possible error or confidence in doing that on account of not having a complete instant-by-instant or mathematically continuous record of the signal function. A microsecond is quite short but still not infinitesimal! :-)

Interestingly you can ignore/delete the lower frequency series terms and thus concentrate more on higher-rate-of-change elements. Adobe Photoshop say, or the software that looks for a tank shape in a missile camera view, will perform this when doing edge detection or edge contrast - because an edge is where some characteristic of the image changes very quickly compared to the entirety of the scene ( higher frequencies disclose derivatives ). You wouldn't care what the DC component is for that, and BTW we are considering frequency in the sense of repetition over distance rather than in time.

Quote:

Another aside is that the human ear-cochleae-brain system essentially transforms the sound compression waves into a pitch spectrum ('Fourier transform') that we then experience as pitch, tone, relative loudness, and other qualities.

A great example of many signal processing features. Many artificial hearing aids, certainly the more expensive and smaller ones, function like a stereo system's 'graphic equaliser' panel to either block inputs or enhance frequencies. Especially in the higher frequency range as that is where all the interesting discrimination is, like what makes a certain person's voice recognisably individual or distinct, plus directional localisation of the source. There are parallels in our visual systems too, like groups of cells in the occipital cortex that trigger only on certain things : like a fast moving object appearing in the visual field periphery that causes you to pull your head way from that side and shut your eyes tight - imagine the vetoes on that!

Quote:

ps: Sorry for the painful multiple pun on note...

There will be no punishment where no pun is meant .... :-) :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter.Blaise Pascal

## The Fundamental Theorem of

)

The Fundamental Theorem of Calculus or Why You Hate Calculus Part II Or Why Are Integration and Differentiation Inverse Operations?

Differentiation finds the gradient of a curve. Integration finds the area between a curve and the horizontal axis. Not self-evidently inverse/reverse/converse procedures. Well, not upon the same function at least. The basic trick is to think of differentiation and integration as operations which turn one function into another.

Recall that a function f is the totality of all (x, y) ordered pairs, where x is a number in the domain of the function f and y is that number in the range of the function f associated with x. When graphing a function we plot those number pairs on a Cartesian plane ( mind you, there are functions which have many variables in the domain, and many in the range. Letâ€™s not talk of those .... ) We thus can write y = f(x) to indicate that f â€˜transportsâ€™ x to y. With certain restrictions one can define an inverse function, call it f^[-1], which is based upon fâ€™s behaviour so that x = f^[-1](y) is a sensible statement : meaning that f^[-1] â€˜transportsâ€™ some y back to the x from whence it first came via the action of f. When this is allowed it is a symmetric situation though, so which function has the superscipt/exponent of -1, reminding us of the numerical idea of â€˜reciprocalâ€™, is a matter of taste. I could give f's inverse the name g for instance, and simply say that f and g are each other's inverses. This you would write as f = g^[-1] and g = f^[-1]. Especially note that f^[-1] does not denote the function which forms the algebraic multiplicative inverse that we call division, although conceivably the manner in which the functions convert x y values might involve that.

So if you have some set/collection of functions then it could be the situation where applying the ( limit based ) processes involved in integration and differentiation might allow you to transfer between one function and another function. Thus it can be the case that differentiating a function h produces the function f, and thus integrating/anti-differentiating f produces the function h. f is therefore called the derivative of h and is thus the function which represents the tangent gradients for each applicable point ( to be discussed ... ) in the domain of h. h will be called the anti-derivative or integral of f. We will show that h ( potentially at least ) represents either a cumulative area sum up to a given point in the domain of f or simply 'that function which when differentiated produces f'. I guess it's an historical thingy, but no one talks of the terms 'anti-integration', 'anti-integral' etc. Whew!! We could show these concepts like so :

where the set has functions as it's members, and thus is not either the domain set or range set of numbers upon which functions are defined to act upon. Again, differentiation and integration are operations upon functions - which is a rather higher level concept than a function associating members of a domain set to those within a range set.

Now weâ€™ve discussed integration which has the following graphical/geometric interpretation :

this is a definite integral, as discussed, with particular choices of the endpoints A and B. Differentiation, which we didnâ€™t discuss, looks like this ( with various annotations for the derivative ) :

Maybe thus it is time to explain differentiation, otherwise this topic will make little sense. I should have perhaps brought it up earlier. I thought we could get away with it, but the square-wave/box-function analysis just done required at least a basic understanding of anti-differentiation .....

The line that typifies the â€˜slope of the curveâ€™ is known as the tangent line. The tangent â€˜just touchesâ€™ the curve at one point. Gradient is generically phrased as being â€˜rise divided by runâ€™ or a ratio of two lengths, the rise being how far up/down one goes and the run being how far along/back. There will be a problem if the run is zero, as that means 'divide by zero' and we will never be doing that in what follows. A rise of zero is OK though. You can satisfactorily analogise/visualise here by thoughts of roads, hillsides, roofing styles and aeroplane flightpaths etc .... if you like. The rise, the run and hence the gradient are signed quantities. By convention - positive is for rise going up and run going to the right, negative is for rise going down and run going to the left. That is, the same sense as the standard Cartesian orientation of x and y axis directions. Differentiation is a limiting process, where we approximate some quantity by successively finer distinctions. To this end I introduce the secant line which is going to be our approximating mechanism :

and youâ€™ll note that the secant intersects our curve at two points, whereas the tangent does so at only one point. The secant line and the tangent line have one point in common â€“ the point at which we wish to evaluate the slope/gradient/derivative. This brings up a very important point : whereas integration involved an entire ( and possibly quite broad, or even the whole ) subset of fâ€™s domain and range, differentiation is concerned almost exclusively with progressively smaller chunks of fâ€™s domain and range. In the strictest sense the Q's only need to be arbitrarily close to P, not arbitrarily far away from P. So you could term differentiation as a local procedure with integration being of a global nature. Although I've drawn Q on the right side of P, we need to allow Q to come from the left as well. Regardless of which side Q comes from the secant gradient limit must be the same. Note that while both the quantities |f(b) - f(a)| and |b - a| go towards zero the ratio of the two may not - indeed that is the key point in finding a 'rate of change'.

As with integration you can work from the basic definitions : pick a function, pick a point, form the algebraic quantity that is the secant gradient, express that in terms of the distance |b - a| and study what limit value that secant gradient approaches as |b - a| goes toward zero ( but is never zero ) - the whole epsilon thingy. OR you can use general theorems which have been deduced from that base level and work with those - because life is too short! :-)

Now the limit of the secant gradient can't always be reached. The main cause of failure is discontinuity of the function, which generally means some 'gap' in the curve and there are no P's or Q's to speak of in the gap that we can examine. But there are other reasons like the function not being defined at all on some interval, or the 'pointy-ness' mentioned earlier that causes the left sided gradient limit to be not equal to the right sided gradient limit. The endpoints of intervals can be an issue too, say where you have a function defined on a closed interval - one that includes the endpoints in the set - like [A, B] but not outside that ie. if x B. We use square brackets '[' or ']' when specifying an interval to indicate if an endpoint is included, and round ones '(' or ')' if an endpoint isn't. Then if you want the derivative at A you can't form any left sided limits as the function has no values there ie. there is no Q ( you might get a derivative at any particular point just to the right of A as there will be some points between such a chosen point and A to work with ). Likewise for any right sided limits for B. Thus the function may have it's derivative defined for the interior of the interval, meaning at every point except the endpoints ie. (A, B). Whew!! :-)

Now take some function for which I can form the integral as follows :

h(t) represents the area under the curve from x = 0 to x = t. Now as f(0) = 0 then h(0) = 0 too, as there is no initial area under the curve. So h(2) say would be the area, as calculated by the integral, b/w the curve f(x) and the x-axis bounded by x = 0 and x = 2. Just imagine sliding the x = t point on the x-axis back and forth and thinking how h(t) will vary as you do that. As the curve is drawn then h(t) will increase as t goes to the right - as more area is covered by [0, t] - and h(t) will decrease as t goes back to the left.

Here is the key point : the greater f(x) is at some point the more area b/w it and the axis will be added into the integral that defines the function we have called h(t). We showed this earlier as those tall thin boxes contributing to the integral sums via the Reimann method. Hence how quickly h(t) changes is proportional to f(x), as if f(x) is low then only a little area adds in whereas if f(x) is high alot of area is added - for equal ( small ) steps along the x direction. If f(x) suddenly went to zero at some point then no area is involved and h(t) would not increase and would reflect the cumulative area under the curve up until the point when f(x) did go to zero.

Algebraically if dt is the little extra bit of increase in t then :

the increase in area = h(t + dt) - h(t) ~ f(t) * dt

or f(t) ~ [h(t + dt) - h(t)]/dt = [h(t + dt) - h(t)]/[(t + dt) - t]

thus f is about equal to the rise/run on the curve of h(t). If you then take the limit of the rise/run ratio as dt goes to zero then f becomes exactly equal to the rate of change of h. Hence the function f is the derivative of the function h, where h is the function representing f's integral ...... TA DA !!! :-) :-)

Note, yet again, that The Fundamental Theorem Of Calculus does not speak of the derivative and the integral of a single function - it says the derivative of one function h is another function f, and the integral of f is thus h.

Go back to the supermarket queue if that helps. How quickly the running total of your grocery bill increases, item by item, depends on the price of the items as they are scanned. Add in a cheap item then the total increases a small amount, add in an expensive one then the total jumps by a fair bit when that item is scanned through .... :-)

Cheers, Mike.

( edit ) The reason I first discussed functions and their inverses was to (a) emphasise such detail about that, as was not previously mentioned but I reckon we'll need and (b) to have something to contrast against - functions operating on numbers as opposed to operations on functions.

( edit ) Try this : a sequence of N ( = 10 ) numbers, a(n) = {1, 5, 7, 2, 0, 6, 3, 1, 1, 5} where n = 1 ...... 10

do the partial sums S(n) = SUM(1,n)[a(n)]

S(1) = 1

S(2) = 1 + 5 = 6

S(3) = 1 + 5 + 7 = 13

S(4) = 1 + 5 + 7 + 2 = 15

S(5) = 1 + 5 + 7 + 2 + 0 = 15

S(6) = 1 + 5 + 7 + 2 + 0 + 6 = 21

S(7) = 1 + 5 + 7 + 2 + 0 + 6 + 3 = 24

S(8) = 1 + 5 + 7 + 2 + 0 + 6 + 3 + 1 = 25

S(9) = 1 + 5 + 7 + 2 + 0 + 6 + 3 + 1 + 1 = 26

S(10) = 1 + 5 + 7 + 2 + 0 + 6 + 3 + 1 + 1 + 5 = 31

obviously the difference b/w S(n+1) and S(n) is a(n+1) - the latest term to be added to the running total from n = 1. So

a(n+1) = [S(n+1) - S(n)] = [S(n+1) - S(n)]/1 = [S(n+1) - S(n)]/[(n+1) - n] = {rise in S(n)} / {run in n}

look familiar? :-)

So the basic reason why integration and differentiation are opposite operations on functions is the same as why addition and subtraction are opposite operations on numbers. With our sequence & series example here we used integers to index the values, with integration/differentiation we use real numbers to index the variables.

I have made this letter longer than usual because I lack the time to make it shorter. Blaise Pascal

## Okey dokey, another Fourier

)

Okey dokey, another Fourier co-efficients example, a triangle wave :

which upon suitable analysis has the following line of C(k) lollipops :

which are all real, non-negative and more savagely diminishing as k increases than was the case with the box function. C(0) = 1/2 too. We can likewise animate accumulating contributions to the waveform from these co-efficients :

which are cycling from k = 0 to 21 and back down again to k = 0. You get the triangle wave forming real quick. These partial sums are of the form :

given that the 'total size' of the function is about order 1, then a number like (4/81*PI^2) ~ 0.0012 ~ 0.1% is quite a small correction ( cosine of anything has magnitude no greater than 1 ) and occurring only in the first handful of terms. The triangle being continuous ( though not everywhere differentiable ) whereas the box is not continuous everywhere on it's quoted domain. BTW if a function fails to be continuous and/or differentiable even at only a single point on it's domain, it is awarded the relevant title 'discontinuous' or 'non-differentiable' anyway - a strict club indeed !

Box Function Redux I didn't mention the partial sums for you to look at and compare with the triangle function's :

Unfortunately I drew the box cliff/sides as an unbroken vertical line, I really should have graphed as follows :

to emphasis the gap, for at t = +1/2 or -1/2 you can only have one point above or below. You can choose the cliff top, bottom or half way up ( or whatever ) say for f(1/2) and f(-1/2) - but not the whole cliff face at once! Otherwise we abrogate the definition of a function, meaning you can't have a plurality of values in the range for a single value in the domain. Sorry for my bad in the graphing, though it wasn't relied upon in the subsequent calculations.

However this does raise a rather interesting point. If one takes the C(k)'s as analysed for the box function and plug them back into the synthesis equation :

and then use that to evaluate f(1/2) and f(-1/2), one gets the answer f(1/2) = f(-1/2) = 1/2 regardless of the original choice of f(1/2) or f(-1/2) in the definition! So the initial placement of those two points is really irrelevant amongst the continuous infinity of points in the time domain, in fact you could even refuse to define the function at all with those particular two points ie. exclude them from the domain. The process - from function prescription to analysis of co-efficients to synthesis leading back to the function again - somehow 'knows' what the average value of the function at t = -1/2 or 1/2 'should' be : halfway up the cliff. Apparently this applies for other similiar 'gap' examples, and there is an awfully deep reason why this should be so ( which I haven't uncovered yet ) but my guess is related to integrals being a global process on a function ........ ;-)

[ Now this segway's into the another idea which is more relevant to what happens at E@H : what if we don't have a continuous record of f(t) over all the time domain? Mathematically we might hypothecate a continuous infinity of points in a finite range ( eg. the real number interval [0,1] ), but in reality that's not going to be what's measured. Firstly we can't store an infinite number of values and secondly we can't 'slice' time into infinitely narrow intervals. What we will get is a sampling of a function : the photodetectors will average out a lot of photons received within some short time period AND while that is an 'analogue' signal ( to the extent that quantum mechanics truly allows ) we will still be converting that to a digital representation with a granularity determined by the chosen bit length. In short our fancy integrals don't fly. What is then used goes generally by the moniker of the Discrete Fourier Transform ( though some say is more accurately described by the phrase 'Discrete Fourier Series' ), these days enacted/implemented by the very efficient Fast Fourier Transform algorithm. But I'll get to that after some more preparation on my side ... :-) ]

Now earlier we looked at Rayleigh's Identity :

I won't bore you with the proof detail, the key step of which is interchanging integration with summation, but discuss further. Physically ( say for a light signal composed of photons ) it says that within one cycle of a periodic wave the power ( energy received per that time interval ) is the same whether you add it up in the time domain or the frequency domain. Let's illustrate/imagine with a thought experiment :

You are in a dark room with a photodetector measuring some periodic signal delivered to you in the form of a beam of light coming from an adjacent room through a suitable hole in one wall. I am in that other room which a bunch of apparatus, lasers etc. You can analyse the signal into it's Fourier co-efficients, presumably to a precision/extent limited by our mutual exhaustion and other practical factors - no matter. Now you wouldn't know it, but I'm in that adjacent room with a bunch of very special lasers, mirrors and whatnot - as many as I can afford and/or fit in - and that each laser is producing a 'pure tone' related to the fundamental frequency of the periodic signal you are studying ie. base plus harmonics. I can modulate the power and the relative phases of each of those lasers and shine the summation ( light combines linearly ) through the wall hole to reach your photodetector. Your calculated Fourier co-efficients will directly relate, with only questions of scaling and translation, to my laser and apparatus settings! Here's what Rayleigh's Identity means : you can count the photons moment to moment and sum to get the LHS, whereas I can add up those same photons but laser by laser and get the RHS!! The total photon energies involved going through the hole b/w the two rooms is simply accounted for in a different manner either side of the wall. This goes back to the vector space analogy too - are you looking at the variation in some total vector length ( LHS of Rayleigh's Identity using photodetectors ) OR or am I following that vector's component values ( RHS of Rayleigh's Identity using laser settings ) ?? As the RHS is a sum of squares of component 'lengths', albeit infinite in number, then Rayleigh's Identity is really Pythagorus' Theorem on infinite dimensional steroids! :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter. Blaise Pascal

## Box Function Redux Redux I

)

Box Function Redux Redux I should have been more explicit on the f(1/2) story. If t = 1/2 then cos(PI/2) = cos(3*PI/2) = cos(5*PI/2) = ..... = 0 and same for t = -1/2, that is odd multiples of PI/2 is where the cosine function crosses the horizontal axis. That makes all the terms in the Fourier series zero, except the first which is the constant term of value 1/2. I guess the reason isn't so mysterious ( though the exact proof might be ) - as the constant DC term is the function's average then altering the placement of a few points out of the infinitely many used on the interval of integration ( that derived said constant term ) makes no difference to an average. This rather emphasises the idea that an integral is a 'broad look at' a function over an interval*.

The Story Thus Far With the ultimate aim of understanding signal processing, we started by considering how to break apart, in a linear fashion, a given function into sub-parts. We discarded the power functions as too unruly to use for periodic behaviours. The obvious, but none-the-less quite bright, idea of using periodic functions as sub-units came along and thus the sine and cosine were chosen - which are well understood and have a long pedigree. Being based on that simplest of geometric shapes ( the circle ) you get regularity 'for free' so to speak. These were then combined into the semantically and algebraically tricky - have just a glance at the co-efficient derivations I parked in some links earlier, they certainly made me lollygag a bit in preparation & I never got them right first time - but highly useful new type of number called complex. Try thinking of complex numbers as an abstraction that allows the messy stuff to be left in the engine room while you steer the ship on the bridge. The complexity never leaves ie. no free lunches ( & no pun intended ) but it's managed at a lower level while you get to write and manipulate high level concepts with a concise notation. One should never underestimate the cognitive value of a good notation, we humans seem to need words to express concepts and in math the notations form the 'words'. Then we talked of limits and particular applications of that, specifically integrals which are one way of looking at the totality of some function. Then vectors and the spaces they live in were introduced for the purposes of analogy, with the great concepts of orthonormality, linear independence and projection. All along we kept refining the detail of how to represent functions as series/sums of (co)sinusoids. Then we talked of periodicity in general and the 'one period but many frequencies' rule, finally getting the explicit expression to use when calculating those all important Fourier coefficients with a particular function presented for analysis into sub-components. We did eventually get the rubber to meet the road with some examples, which will also have great utility and relevance later on. We had some snacks & detours along the way like the Fundamental Theorem of Calculus, continuity, differentiation, and another important higher level concept : operations upon functions. Here's some key features to highlight/note again :

- the C(k)'s are the multipliers of the oscillating base functions.

- the DC or k = 0 component doesn't itself oscillate but represents the average level about which others do.

- because we deal with real signals, the +ve and -ve C(k)'s are complex conjugates [ so when you add C(k) times it's oscillating complex exponential to C(-k) times it's oscillating complex exponential then you get a real number ].

- lower frequencies paint the 'broader strokes' while higher frequencies fill in the 'finer detail' like sharp edges/cliffs and pointy bits.

- those pointy bits are correctly represented only as the limit of some progressive approximating process.

- if convergence occurs then there will in general be different 'rates of approach' to the limiting value ie. how many terms to use in the partial series to get within some tolerance level. The rate(s) of change of the function ie. derivative(s) determine this.

Now we ought next wander along to the topic of convolution which is an especial way of combining two functions in one domain that leads to their multiplication in the other domain ( that being reached via some Fourier operation ). The way it will be expressed has a nice symmetry, but to do that we need to expand our Fourier representation of functions to a continuous frequency domain ie. beyond integer multiples and deal with non-periodic functions. It is only much, much later that one will see the true reasons for that route. Plus there will be some nice views along the way ..... :-) :-)

However for the moment I thought we'd roll off the mathematical throttle a bit, coast along, and next show some animations that I'm cooking up - a potpourri of my fiddlings with 3dsMax - to illustrate some of the stuff mentioned so far. And since you're all rather quiet listeners, but I know that several dozen are following along ( some have PM'ed me with further encouragement, so thanks for that ) it could also be a good time to invite questions upon the material shown to date. If you think/worry that you might 'pollute' this thread then feel free to fire up another, not that it would worry me either way. :-) :-)

* which is to say that any of the C(k)'s requires an evaluation of an integral involving the signal function over an entire cycle of the signal. Although I solved for the general k'th case, you can still consider each component frequency requiring it's very own integral specified by plugging in a specific value of k where appropriate in the general Fourier coefficient formula. Indeed if you put k = 0 into that formula, then because exp[zero] = 1, it simplifies to the traditional averaging formula :

here the RHS of that last equality ( in red ) is a bit like finding the length of one side of some rectangle by dividing the rectangle's total area by the width along the other ( orthogonal ) dimension : since area = length * width. You can think of the function's values ( height above the horizontal axis ) as a variable length rectangle side - which is why we added up lots of little rectangles with the Reimann method.

Cheers, Mike.

( edit ) Indeed another way of showing this is : the area b/w the curve and the x-axis could be replaced, on equal area grounds, with a rectangle that has one dimension being the height representing the function's average value and the integration interval being the other dimension :

I have made this letter longer than usual because I lack the time to make it shorter. Blaise Pascal

## All this is linear algebra. I

)

All this is linear algebra. I passed my exam with professor Giovanni Prodi, the eldest of the 5 Prodi brothers, all university professors, but that was a long time ago. My brother Franco was Prodi's assistant professor at Pisa, where Ennio DeGiorgi and Enrico Bombieri, now at Princeton Institute for Advanced Study were the stars. But that was a long time ago. Now I am cooperating with people from CERN on a BOINC project including a virtual machine, and also at QMC@home on quantum chemistry. Cheers.

Tullio

## Ok, time for some animations.

)

Ok, time for some animations. I'll post links with the idea of selecting each to open in a new browser tab as you choose/please, rather than force load them all by inline image inclusion -> because the file sizes are up to ~ 4 MB. They are in some perspective view looking upon a segment of the Argand plane. Sort of like looking at a blackboard, as I use the 'third' dimension here.

Rotating complex exponential - say of 'unit' length and frequency 'one' :

..... note the marking of angles as positive for anti-clockwise and negative for clockwise, but any angular value may differ by any multiple of 2*PI when representing the same 'propellor tip' position on the circle.

Now the same but draw the time axis 'out of the blackboard' so the base of the propellor moves along it, and the tip will trace a spiral/helix upon the surface of a cylinder that could be projected out of the board from the unit circle :

.... only two cycles are shown and repeated from t=0 to t=2T, but you could keep going forever in the direction out of the blackboard!

Now for that helix, if you look down along the positive iy-axis ( as if you were sitting on the top rail of the blackboard ) you perceive the horizontal or x-axis component oscillating only - the cosine that is :

and

.... I deliberately didn't rotate the cosine view to match the usual textbook appearance, so as to emphasise the orthogonality of the concept of sine and cosine. Think back to the definition : while the radius is never zero, the cosine measures the x-axis deflection as a signed fraction of the radius and the sine measures the y-axis deflection as a signed fraction of the radius. This also re-asserts the idea that sine and cosine are time/phase shifted versions of the same shape.

Similiarly if you look along the negative x-axis toward the origin ( from the left side of the blackboard ) you perceive the vertical or iy-axis component oscillating only - the sine that is :

and

.... smart punters will note that the cosine looks the same whether we rotate either way, this is what is meant by the cosine being an even function. The sine one way looks like minus the sine the other way, thus sine is deemed as an odd function - because the initial y-axis deflection is upwards for anti-clockwise rotation and downwards for clockwise.

Let's chuck in some

different frequencies now, anti-clockwise rotation, and you'll note that when the blue guy makes two revolutions, the purple guy does one, the green guy does three, and the brown guy four. Time passes the same for all - the tails all proceed at the same rate along the time axis - but the rate of phase/angle change with time varies. This is of course the frequency, as in cycles per second ( or radians per hour, or whatever according to your phase and time units ). Because these frequencies have some integral multiple relationship then they will all come back to knock on the door at some same time, and repeat that co-incidence only whenever the slowest cycling component does so. The

cosine and

sine views aren't as clear ( the propellors overlie one another ) but show the periodicities.

If you're not used to this topic, sines etc then just stare at them for a while. The relationships and the algebra thereof already mentioned will come to view/click. There will be more .... :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter. Blaise Pascal

## I want to illustrate the (

)

I want to illustrate the ( partial ) Fourier synthesis of the box function that we've already analysed into Fourier coefficients :

This animation will take some detail to explain fully. Firstly go back to the ideas of adding and multiplying complex numbers :

z_1 = x_1 + iy_1

z_2 = x_2 + iy_2

gives

z_1 + z_2 = (x_1 + x_2) + i(y_1 + y_2)

which if you think of as an ordinary vector on a two dimensional plane, this idea translates to taking the new total vector as being the one where the base of second was placed at the tip of the first thus :

Now with :

z_1 = r_1 * exp[i * THETA_1]

z_2 = r_2 * exp[i * THETA_2]

then :

z_1 * z_2 = r_1 * r_2 * exp[i * (THETA_1 + THETA_2)]

which doesn't have a convenient 'plain vector' analogy, alas. Now this :

is a sum of products. So for each choice of frequency ( k value ) I multiply the relevant complex exponential ( which will be rotating clockwise or anti-clockwise depending on the sign of k, and the rotation speed is proportional to the magnitude of k ) by the corresponding Fourier co-efficient which is also k-dependent. The DC component for the box function is 1/2 and that can be thought of as 1/2 times exp[0], where exp[0] = 1 means length one and phase zero.

The DC is

animated quite simply with the time axis coming out of the blackboard, where the green circle on the Argand Blackboard is no longer a unit circle - but everything will be scaled OK. This also shows the path traced by the propellor tip ( a non-rotating propellor that is! ). This path is aligned in direction with the x-axis which it should because we are modelling a real number valued signal that has no nett imaginary component. For what follows I'll assume the base/fundamental frequency is one ie. T = period = 1. So k/T = k for simplicity.

This brings us to the k = 1 and k = -1 components which we promised/agreed earlier to add in only when together, and so being complex conjugates the nett would remain a real number. For k = 1 then C(1) = 1/PI = 1/PI*exp[0] so the length is a bit less than the DC part ( as PI > 3 ) and the phase offset is zero, and thus it's propellor is :

C(1)*exp[2*PI*i*t] = 1/PI*exp[0]*exp[2*PI*i*t] = 1/PI*exp[2*PI*i*t + 0] = 1/PI*exp[2*PI*i*t]

For k = - 1 it's nearly the same except that exp[PI*i] = -1, and thus this propellor has to begin with a phase offset of PI radians or 180 degrees around the circle, and it rotates the other way too , so we have :

C(-1)*exp[-2*PI*i*t] = 1/PI*exp[PI*i]*exp[-2*PI*i*t] = 1/PI*exp[-2*PI*i*t + PI*i] = 1/PI*exp[-2*PI*i*(t - 1/2)]

where the t - 1/2 ( for period of 1 ) indicates a lead, or a lag for that matter, of half a cycle. Put

all this together, and here I animate the additions by head-to-tailing the propellors/vectors, so I can look at the path of the tip of the last chained/jointed propellor to follow the total as it evolves over time. You'll note how the contra-rotating propellors conspire by design to keep that tip always upon some direct perpendicular ( to the Argand plane ) line from the x-axis.

The k = +/- 2, +/- 4, +/- 6 .... etc components require no work at all. Herewith includes up to the

k = +3/-3, and then

k = +5/-5 contributions with it's

'sine' or imaginary view ( pretty boring because we are edge on to the real-axis/t-axis plane ) and the

'cosine' or real view ( far more interesting showing some development of the box shape after the few terms I've animated ).

This all rather reminds me of (a) the 'Spirograph' drawing kit/toy I had as a lad (b) a Hewlett Packard plotter my Dad used to output his engineering drawings from a real early version of AutoCAD + SpaceGas ( finite element structural modeller ) (c) epicycles of either Ptolemaic or Copernican construction. Gauss in effect invented his own version of the FFT ( though not calling it that ) when attempting to interpolate trigonometric functions ( for published/printed look up tables ), and when considering asteroid orbits where by interpolate we also mean predict.

Ok, I'll sticky this and start afresh next time with a new thread : onwards and upwards to the continuous frequency domain ..... :-)

Cheers, Mike.

( edit ) I guess it may well be bleedin' obvious to state but : the animations show the progressive tendency toward convergence of the series by the smaller lengths of the oscillating elements, plus you can see how the sum up to a given level acts as the base for further refinement of the curve, also the phases are crucial to making the trend corrections in the right directions, and how the faster oscillating components really do decide upon the finer curve detail as the slow ones just can't move quick enough to do that. All of these issues about lengths/phases/frequencies really do have a correspondence in one's thinking about some real signal analysis problem. Ignoring matters of polarisation, that's all we really receive from distant objects like pulsar systems ... a sequence of time indexed measurements.

I have made this letter longer than usual because I lack the time to make it shorter. Blaise Pascal

## RE: ... I guess it may well

)

Obvious is good! :-)

The animations very nicely show how the convergence improves as you include ever more resultant points to make the inverse transform increasingly more precise.

I also found it rather interesting to note in what way the transforms return an imprecise version of the original when you restrict the number of transform points. I guess that can be thought of as analogous to undamped filtering the original signal...

Another aside is that the human ear-cochleae-brain system essentially transforms the sound compression waves into a pitch spectrum ('Fourier transform') that we then experience as pitch, tone, relative loudness, and other qualities.

Keep searchin',

Martin

ps: Sorry for the painful multiple pun on note...

Powered by: Mageia5

See & try out your OS Freedom! Linux Voice

The Future is what We all make IT [url=http://www.gnu.org/copyleft/gpl.html](GPLv3

## RE: The animations very

)

Yup, applying a frequency filter ie. in this case by only going a certain distance along a series from the start ( so this will be a low-pass filter ), has the effect of altering the back-transform ( you could call that convolution ). Indeed that sequence is key to most signal processing tasks - do the transform, manipulate in the alternate domain, then do the back transform to the original domain.

The 'center of the maze' in this whole topic, which I'm aiming at, is where you take some once-off waveform and then periodise ie. repeat/duplicate indefinitely in the time domain ( by applying a 'delta comb' ) and then you restrict that result by applying a single box function to get back to the original non-periodic shape. Although that seems a long way around to come back home, the effect will be that you gain the ability to sensibly interpolate in situations where you have only a discrete number of measurements. So you can deduce what the in-between points 'ought be' in be in circumstances where you can only approximate the curve. But you have to place limits on the rate of variance of the curve between measured points ie. a high frequency cut-off, or you won't sufficiently narrow the field of function candidates that could be used to interpolate values. This I reckon is the jewel in the crown, as this gives the DFT/FFT and hence one can search for buried signals in real data. Gauss used this to answer questions about orbiting bodies in the solar system ie. where were they earlier and where will they be later on? Admittedly there is some messy detail relating to the possible error or confidence in doing that on account of not having a complete instant-by-instant or mathematically continuous record of the signal function. A microsecond is quite short but still not infinitesimal! :-)

Interestingly you can ignore/delete the lower frequency series terms and thus concentrate more on higher-rate-of-change elements. Adobe Photoshop say, or the software that looks for a tank shape in a missile camera view, will perform this when doing edge detection or edge contrast - because an edge is where some characteristic of the image changes very quickly compared to the entirety of the scene ( higher frequencies disclose derivatives ). You wouldn't care what the DC component is for that, and BTW we are considering frequency in the sense of repetition over distance rather than in time.

A great example of many signal processing features. Many artificial hearing aids, certainly the more expensive and smaller ones, function like a stereo system's 'graphic equaliser' panel to either block inputs or enhance frequencies. Especially in the higher frequency range as that is where all the interesting discrimination is, like what makes a certain person's voice recognisably individual or distinct, plus directional localisation of the source. There are parallels in our visual systems too, like groups of cells in the occipital cortex that trigger only on certain things : like a fast moving object appearing in the visual field periphery that causes you to pull your head way from that side and shut your eyes tight - imagine the vetoes on that!

There will be no punishment where no pun is meant .... :-) :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter. Blaise Pascal