detecting steep rises and drops of data

HOW TO ATTACH YOUR SAMPLE WORKBOOK:

Fast answers need clear examples. Post a small Excel sheet (not a picture) showing realistic & representative sample data WITHOUT confidential information (10-20 rows, not thousands...) and some manually calculated results. For a new thread (1st post), scroll to Manage Attachments, otherwise scroll down to GO ADVANCED, click, and then scroll down to MANAGE ATTACHMENTS and click again. Now follow the instructions at the top of that screen.
Dismiss

Where it's for a thesis, we probably need to be careful that we choose and algorithm and/or programming steps that can be documented in some way (I doubt "some random nobody(s) on the internet told me to do it this way" goes over very well when documenting or defending a thesis).

Do you have a peak identification algorithm in mind? We are usually pretty good at programming a known algorithm into the spreadsheet, but not always knowledgeable in specific fields of study to know what algorithms apply to your specific project.

I'm no expert in signal processing or peak identification, but I am aware of some principles of this kind of analysis. From what I have seen, peak identification in a signal may include:

1) distance from baseline (which means needing a baseline detection algorithm) should be above a threshold value before deciding that a portion of the signal represents a peak.
and 2) slope ought to be above threshold in order to separate a peak from baseline drift.

I find that, whenever I have had to do something like this, I have almost always started with a column calculating slope. Perhaps slope alone can do both baseline detection (whenever slope is within threshold of 0 with some way of distinguishing 0 slopes at baseline and 0 slopes at the top of peaks) and peak start/stop detection (peak start is when slope goes from near 0 to above some positive value. peak end is when slope goes from some negative value below a threshold and then becomes 0/baseline). (Edit to add -- I see that someone else had a similar idea to this one).

Does that help think through some basic ideas?

I need to gather all the data, which lie in between the data of the start and end of one step because I need to sum them up for further calculation .
The hole process is about transforming pressure into force.

That sounds like, after identifying each peak, you need to integrate each peak. Usually, I would expect to see a baseline algorithm of some kind that numerically describes the baseline at the bottom of the peak. Assuming the baseline is approximated as straightlines, one usually ends up, then, with a series of trapezoids that you can find the area of, then add all those trapezoids up.

It appears that we are back to the peak detection part of the algorithm. Reverse engineering the existing peak detection algorithm suggests that the current peak start/end algorithm is:

Peak start: look at a 12 or so point window and find windows where only the last point's change is above threshold
peak end: look at a 12 or so point window and find windows where only the first point's change is above threshold

In the example given, the top of the peak (where the change is "small") is broad enough to cause a false positive with this peak detection algorithm. I expect that changing the "width" of the window (18, 20, 30 points instead of around 12???) will resolve this one.

One parameter I see in the peak detection software I use is a "peak width" parameter that tries to give the software some idea of how narrow/wide the peaks are expected to be. Right now, your "peak width" is hard coded in the spreadsheet formulas. Adding another parameter that makes "width" a user selectable variable adds a layer of complexity to the programming (probably an OFFSET() function). Is it worth the effort to add a layer of complexity to the formulas, or is this something that you can change in the formulas manually?

At this point, it feels to me like we are back to peak detection. You might need to come up with a good, representative sample of the kinds of peaks you expect to see (a representative sample of widths with and without shoulders, since your latest example also has a shoulder on the upslope of the peak) so that the peak detection algorithm can be thoroughly developed and tested.