+ Reply to Thread
Results 1 to 6 of 6

Working out if data is regularly periodic

  1. #1
    Registered User
    Join Date
    10-09-2019
    Location
    London, England
    MS-Off Ver
    365
    Posts
    3

    Working out if data is regularly periodic

    I have a month-by-month dataset, with various periodic entries (i.e. some on a regular monthly basis, others with regular quarterly entries and others on a semi-annual basis). I'm trying to determine what their period is automatically (the full dataset is several thousand rows).

    I have seen ways of doing the reverse (i.e. automating input of periodic data by using a mod function to compare a month counter with the desired periodicity), but I am struggling to see how to achieve the determination of regular period. I also had a thought about using counting blank cells in comparison to full cells, but this doesn't distinguish between regular and irregular interval values.

    Any suggestions would be greatly appreciated!

    Many thanks,
    Attached Files Attached Files

  2. #2
    Forum Expert
    Join Date
    10-15-2018
    Location
    MA, USA
    MS-Off Ver
    2010, 2019
    Posts
    1,616

    Re: Working out if data is regularly periodic

    Format column-U as "Number"

    In U5 copied down to U9:
    Formula: copy to clipboard
    Please Login or Register  to view this content.


    This gives the correct answer for monthly, quarterly and semi-annually and for non-periodic(for which U8 will be non-integer). The result for annual is 9 and is incorrect because your chosen 18 month time period is not a multiple of years.

    Hope this helps
    Geoff

    Did I help significantly? If you wish, click on * Add Reputation to say thanks.
    If your problem has been resolved please select ?Solved? from the Thread Tools menu

  3. #3
    Registered User
    Join Date
    10-09-2019
    Location
    London, England
    MS-Off Ver
    365
    Posts
    3

    Re: Working out if data is regularly periodic

    Hi -

    Thanks, however, that formula doesn't capture regular periodicity - if you, for example, move one of the quarterly values forward by a month, or postpone it by a month, you still get 3.0 - as the total number of filled cells is still the same. I accept that the non-periodic one with a non-divisible number of filled cells provides a non-integer, but if you simply delete one of those cells, then there are 6 filled cells and you get an answer of 3.0, which would imply quarterly values, even though in reality these are not equally distributed.

    Any additional thoughts?

    Thanks,

    Joachim
    Last edited by Joachim_91; 07-02-2020 at 05:23 AM.

  4. #4
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,829

    Re: Working out if data is regularly periodic

    How much of this is specific to Excel and how much is a broader algorithm development question -- independent of programming language? If you know the algorithm you want to use and can help us understand it, we should be able to help you program that algorithm into the spreadsheet. If you are more in the algorithm development stage, we can probably help with that, too, but it is a little bit different question.

    If this is more about algorithm development, it seems to me that we need more details about what you are trying to do. For example:
    if you, for example, move one of the quarterly values forward by a month, or postpone it by a month, you still get 3.0 - as the total number of filled cells is still the same.
    implies that you will tolerate no deviation from a pattern in detecting that pattern. However, If I saw a set where all of the payments but one were quarterly, and the exception was only 1 month early/late, I would be inclined to call that a quarterly pattern, but your statement suggests that even one payment being one month early/late makes an otherwise quarterly pattern have no pattern. If that's what you want, that's fine, but I think it will be important to be very clear about what you want to do in order to develop a robust algorithm (and then program).

    If there are only a few very specific patterns that need to be detected, I could see this somewhat "brute force" algorithm working (in more outline/pseudocode form):

    1) a sign function to detect which months have an entry, then join into a single "binary number" string. CONCAT(SIGN(range)) kind of thing.
    2) in a convenient place in the spreadsheet, build a lookup table with the desired "binary number" patterns you want to detect and the pattern they represent.
    3) use the result of (1) in a lookup function to return the correct pattern output. If the result of (1) is not found in the lookup table, then return a no pattern indicator. VLOOKUP(result of 1,lookup table,2,FALSE)

    That should work as long as the number of patterns you wish to detect is small and your matching algorithm is exact. It will probably become unwieldy if you have a lot of different patterns to detect and/or if your desired matching algorithm is more fuzzy.
    Quote Originally Posted by shg
    Mathematics is the native language of the natural world. Just trying to become literate.

  5. #5
    Registered User
    Join Date
    10-09-2019
    Location
    London, England
    MS-Off Ver
    365
    Posts
    3

    Re: Working out if data is regularly periodic

    Hi MrShorty -

    Thanks, that's some helpful thoughts over which to mull. I've spent today working up a solution, and think I found something that can work:

    1. Create a counter that starts counting blanks once it detects a value. It will then stop counting and reset once another value is detected in the same row
    2. Check what maximum value the counter reaches each time before it resets.
    3. Calculate the variance of this flow. If the variance is 0, that therefore means the counter reaches the same value each cycle, and so a particular row has constant period (with the counter number representing said period)
    4. If variance is not zero, then your row doesn't have a regular period. To your point about small deviations however, a very low variance (such as 0.05) may indicate that though a set of payments tends to be made on a quarterly basis, there was *an* instance of a irregular late payment
    5. Greater variance can indicate changing periodicity, or non-periodic rows

    Think that might work, but welcome additional thoughts!
    Attached Files Attached Files

  6. #6
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,829

    Re: Working out if data is regularly periodic

    It looks fine to me. My mentor commented in some of his programs that he sometimes programmed things to be easy for the programmer even if they weren't necessarily the easiest for the computer. If you understand what you did well enough to use it, change it (when needed), and adapt it to other scenarios, then it might just be good enough, and might be better than someone else's "better" solution that you don't understand or cannot change or adapt.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Taking data from regularly updated table
    By eveza in forum Excel Formulas & Functions
    Replies: 2
    Last Post: 02-16-2019, 08:31 PM
  2. Transferring data from a closed workbook which is updated regularly
    By sdhutty in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 11-16-2016, 05:01 AM
  3. Importing Data from a regularly updated CLOSED excel workbook
    By sdhutty in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 11-10-2016, 06:31 AM
  4. Automated periodic data capture
    By Newton51 in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 02-28-2011, 05:05 PM
  5. Regularly updated time display: not working
    By madhg in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 06-22-2009, 03:43 AM
  6. How to regularly import data from Oracle database without linking the table?
    By shadestreet in forum Access Tables & Databases
    Replies: 4
    Last Post: 05-04-2009, 10:38 AM
  7. [SOLVED] Retaining data permanently from periodic changing data in workshe.
    By Exel User in forum Excel Formulas & Functions
    Replies: 1
    Last Post: 03-05-2006, 10:22 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1