+ Reply to Thread
Results 1 to 13 of 13

How to discriminate values between spikes?

  1. #1
    Registered User
    Join Date
    06-05-2015
    Location
    USA
    MS-Off Ver
    2003, 2016
    Posts
    24

    How to discriminate values between spikes?

    I have values that represent the temporal complexity of each proceeding video frame. I am trying to remove all the visually-duplicate ones but the trouble is they aren't exact duplicates so they need a threshold which itself is a problem because setting it high enough to kill every duplicate will also kill legitimate frames with low complexity such as slow panning or cloud movement.

    Luckily each animated video has patterns. Some scenes use the full FPS and have no duplicates, some are half the framerate, some are 1/3 of it. To deal with the scenes where every second frame is a duplicate, I did =A1/A2 formula which I think effectively discriminates the bottom value between each spike:

    35.9146 6.475182548
    5.5465 0.179759003
    30.8552 10.61629507
    2.9064 0.111036401
    26.1752 4.630484008
    5.6528 0.216141811
    26.1532 5.487452791
    4.766 0.209058011
    22.7975 7.026073289
    3.2447 0.190551976
    17.0279 5.253254766
    3.2414 0.200646248
    16.1548 5.420346262
    2.9804 0.228211764
    13.0598 11.73492677
    1.1129 0.159372762
    6.983 3.499373591
    1.9955 0.024428432
    81.6876 29.42318914
    2.7763 0.121364592
    22.8757 18.77519698
    1.2184 0.029882227
    40.7734 7.880288359
    5.1741 0.136040238
    38.0336 20.69743143
    1.8376 0.045935177
    40.0042 17.44927157
    2.2926 0.039427047
    58.1479 45.35717629
    1.282 0.021224533
    60.4018 20.98960976
    2.8777 0.057400397
    50.1338 11.24858085
    4.4569 0.096459049
    46.2051 22.12675989
    2.0882 0.052623223
    39.6821 20.97362579
    Anything below 0.5 up here is obviously a bottom spike. But my problem is with scenes that have one unique frame in every third frame. A floor-floor-spike pattern, which my current formula will fail on. Observe:

    43.4271 8.137596972
    5.3366 1.021319758
    5.2252 0.114447671
    45.6558 12.40714169
    3.6798 1.045338333
    3.5202 0.075928235
    46.3622 23.7681739
    1.9506 0.5542422
    3.5194 0.096100967
    36.6219 10.38388908
    3.5268 0.541460045
    6.5135 0.188867174
    34.4872 7.103439753
    4.855 1.155017367
    4.2034 0.13446405
    31.2604 3.216123623
    9.7199 2.13024897
    4.5628 0.132141699
    34.5296 9.93457433
    3.4757 1.112508802
    3.1242 0.087898917
    35.5431 6.515692026
    5.455 1.538179562
    3.5464 0.184684286
    19.2025 3.585700148
    5.3553 1.259122543
    4.2532 0.062959164
    67.5549 8.260766954

    I need the low values between each spike to be 0.5 or less or the threshold won't recognize them as duplicates.

    Anyone have the obvious solution my feeble-minded head is not comprehending?
    Last edited by Inds; 06-05-2015 at 11:10 PM.

  2. #2
    Forum Moderator Glenn Kennedy's Avatar
    Join Date
    07-08-2012
    Location
    Digital Nomad... occasionally based in Ireland.
    MS-Off Ver
    O365 (PC) V 2406
    Posts
    44,423

    Re: How to discriminate values between spikes?

    44 views and no replies.... I don't understand what I am seeing and what we are meant to do with it!! You have two columns of numbers. Start by telling us what they are, and take it from there....
    Glenn




    None of us get paid for helping you... we do this for fun. So DON'T FORGET to say "Thank You" to all who have freely given some of their time to help YOU

  3. #3
    Registered User
    Join Date
    06-05-2015
    Location
    USA
    MS-Off Ver
    2003, 2016
    Posts
    24

    Re: How to discriminate values between spikes?

    I said outright what they are in the very first sentence. They are the temporal complexity of proceeding video frames. I want the small numbers between the big numbers nullified. Notice the pattern? Big number, small number small number, big number.

    I realize my description might've been confusing because you guys are video experts as much as I'm an excel expert. So if anything's confusing I'll gladly rephrase.

    The second column is my attempt to isolate the bottoms of each spike by dividing one value with the proceeding one, that way I can delete any value that's proceeded by one with twice the value. The issue is this only works with one small number surrounded by a spike, not two small values.

  4. #4
    Forum Expert
    Join Date
    09-01-2012
    Location
    Norway
    MS-Off Ver
    Office 365
    Posts
    2,844

    Re: How to discriminate values between spikes?

    Post a workbook instead where you include some sample data and the expected results. It will make it much easier to help you.
    <----- If you were helped by my posts you can say "Thank you" by clicking the star symbol down to the left

    If the problem is solved, finish of the thread by clicking SOLVED under Thread Tools
    I don't wish to leave you with no answer, yet I sometimes miss posts. If you feel I forgot you, remind me with a PM or just bump the thread.

  5. #5
    Registered User
    Join Date
    06-05-2015
    Location
    USA
    MS-Off Ver
    2003, 2016
    Posts
    24

    Re: How to discriminate values between spikes?

    Ok, column on left is a sample of the data and column on right is what I want it to be. Doesn't have to be precisely 0 just distinguished from the spike values they're surrounded by.

    43.4271 43.4271
    5.3366 0
    5.2252 0
    45.6558 45.6558
    3.6798 0
    3.5202 0
    46.3622 46.3622
    1.9506 0
    3.5194 0
    36.6219 36.6219
    3.5268 0
    6.5135 0
    34.4872 34.4872
    4.855 0
    4.2034 0
    31.2604 31.2604
    9.7199 0
    4.5628 0
    34.5296 34.5296
    3.4757 0
    3.1242 0
    35.5431 35.5431
    5.455 0
    3.5464 0
    19.2025 19.2025
    5.3553 0
    4.2532 0
    67.5549 67.5549

  6. #6
    Forum Moderator Glenn Kennedy's Avatar
    Join Date
    07-08-2012
    Location
    Digital Nomad... occasionally based in Ireland.
    MS-Off Ver
    O365 (PC) V 2406
    Posts
    44,423

    Re: How to discriminate values between spikes?

    In B1 and drag down=if(a1<8,"",a1) will do i, but there isn't any science behind the assignation of the cut-off point. One could also do it on the basis of the mean minus 3 or 4 SD of the peaks, but we'd need to know a bit more about your data. Can you tell a priori if a dataset is going to be Hi-Lo-Hi-Lo or Hi-Lo-Lo-Hi-Lo-Lo?

  7. #7
    Registered User
    Join Date
    06-05-2015
    Location
    USA
    MS-Off Ver
    2003, 2016
    Posts
    24

    Re: How to discriminate values between spikes?

    If the proceeding value is at least twice the value then it counts as a spike, but I need it to look at a longer range of cells to deal with the Hi-Lo-Lo-Hi cases. No I can't tell which scenes will be Hi-Lo-Hi or Hi-Lo-Lo-Hi. I mean I can but this video has 30,000 frames thus 30,000 values in the log so it would take hours and hours to separate them all manually. That's why I need a formula to automatically deal with this.

  8. #8
    Forum Moderator Glenn Kennedy's Avatar
    Join Date
    07-08-2012
    Location
    Digital Nomad... occasionally based in Ireland.
    MS-Off Ver
    O365 (PC) V 2406
    Posts
    44,423

    Re: How to discriminate values between spikes?

    Yes, I see the problem: in your first post you had a H-L-H-L sequence:

    16.1548
    2.9804
    13.0598
    1.1129
    6.983
    1.9955
    81.6876
    2.7763

    You need to assign the line in bold, above, as a spike. It's night time here (I'm currently in Poland). I'll ruminate on this overnight and, if no-one else has sorted you out; I'll be back sometime during the EU's morning.

  9. #9
    Registered User
    Join Date
    06-05-2015
    Location
    USA
    MS-Off Ver
    2003, 2016
    Posts
    24

    Re: How to discriminate values between spikes?

    Yes, there are 3 different patterns. Ones with no fluctuation (no spikes) because the scene has no duplicate frames. Ones with a Hi-Lo pattern and ones with a Hi-Lo-Lo pattern.
    I can post the whole log if it'll be helpful.
    Thanks for the help.
    Last edited by Inds; 06-06-2015 at 06:46 PM.

  10. #10
    Registered User
    Join Date
    06-05-2015
    Location
    USA
    MS-Off Ver
    2003, 2016
    Posts
    24

    Re: How to discriminate values between spikes?

    Buummpp!!!

  11. #11
    Forum Moderator Glenn Kennedy's Avatar
    Join Date
    07-08-2012
    Location
    Digital Nomad... occasionally based in Ireland.
    MS-Off Ver
    O365 (PC) V 2406
    Posts
    44,423

    Re: How to discriminate values between spikes?

    I had posted last night, asking you to post the log... as this is either much simpler than I think, or much more complicated than I fear - but I don't know which... However, the post hasn't appeared. Thank you... free wifi in Krakow.

  12. #12
    Registered User
    Join Date
    06-05-2015
    Location
    USA
    MS-Off Ver
    2003, 2016
    Posts
    24

    Re: How to discriminate values between spikes?

    I thought you just wanted a sample. Here's the whole log.
    Attached Files Attached Files

  13. #13
    Registered User
    Join Date
    06-05-2015
    Location
    USA
    MS-Off Ver
    2003, 2016
    Posts
    24

    Re: How to discriminate values between spikes?

    bbuuummmppp

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] Need an excel formula that can weed out or smooth spikes
    By mburke in forum Excel Formulas & Functions
    Replies: 8
    Last Post: 11-19-2013, 10:53 AM
  2. Formual that provides a monthly average smothing spikes
    By mburke in forum Excel Formulas & Functions
    Replies: 1
    Last Post: 11-18-2013, 03:38 PM
  3. Count, add, discriminate and give me an answer
    By LawrieAJ in forum Excel General
    Replies: 4
    Last Post: 07-13-2013, 08:24 AM
  4. Find max value in multiple spikes in a data set
    By bmcfiv in forum Excel General
    Replies: 1
    Last Post: 07-23-2012, 12:35 AM
  5. Detect and ignore data spikes
    By loser in forum Excel General
    Replies: 1
    Last Post: 03-14-2011, 08:12 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1