+ Reply to Thread
Results 1 to 6 of 6

Problem: Finding patterns in a table

  1. #1
    Registered User
    Join Date
    02-12-2023
    Location
    Praha
    MS-Off Ver
    2013
    Posts
    12

    Problem: Finding patterns in a table

    Hello!
    Thank you all for YOUR HELP in advance. Your assistance has been priceless. Unfortunately, I have encountered another problem that I'm unsure how to deal with.
    Attached to this post is a file containing a small fragment of a measurement table showcasing changes in temperature for a specific physic-biological process under study. The complete table has dimensions of 600 columns and 1500 rows. For a better understanding of the problem, the table only includes differences in temperature changes, either as an increase by a certain number of degrees [C] or a decrease in temperature [C].

    OVERALL DESCRIPTION:
    ? Rows indicate measurement points: from M1.1-M13.1 to M1.4-M13.4
    ? Columns represent measured values: S1-S50
    ? The numbers you are looking for are cell values marked in red.
    ? The desired changes in temperature values (red cells) are accompanied by:
    1. Constant temperature increases, such as configurations of temperature increase by 1 degree (I marked these cells with a yellow color and bold black frame). The searched value corresponds to repeated constant configurations of temperature increase, for example: 0/1, 0/2, 0/3, 0/4, 0/5, etc.
    2. Constant temperature drops, such as configurations of a temperature drop of 1 degree (I marked these cells with a bold black frame). The searched value corresponds to repeated constant temperature drop configurations, for example: 1/0, 2/0, 3/0, 4/0, 5/0, etc.
    SAMP1.jpg
    EXAMPLE:
    Here's an example of the searched value: The temperature change between the measuring points M3.1 and M2.1 in column S14 corresponds to a temperature increase by 0/4 degrees [?C]. This increase is accompanied by a sharp temperature drop 7/0 (column S15) and a small temperature increase 0/1 (column S13) in the adjacent cells.

    PROBLEM DESCRIPTION:
    1. How to search the table and determine the potential set of searched cell values (cells marked in red or their location in the table) based on known patterns of changes in measured temperature values, for example: temperature increase with the configuration 0/1 or temperature decrease 1/0?
    OR
    2. How to automatically determine potential patterns based on already processed data? I have a part of the table where I have already determined the searched values using analytical calculations, rather than patterns of temperature increase or decrease. Analytical calculations are complicated, time-consuming, and I would like to avoid them.

    I am asking for any idea how to solve the problem. If you have any questions, please contact me. THANK YOU VERY MUCH IN ADVANCE FOR YOUR INTEREST AND YOUR HELP!

    Mark
    Attached Files Attached Files
    Last edited by MarkSSS; 05-26-2023 at 07:45 AM.

  2. #2
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,829

    Re: Problem: Finding patterns in a table

    I am unable to understand your pattern detection logic. It might help you think of programming strategies, but here's what I've done so far.

    One programming strategy I use for this kind of thing is to reduce each possible pattern to a unique number or text string. Your example shows you taking a 2x3 block of cells and comparing the top row to the bottom row and determining if the top row values are larger, the same, or smaller than the bottom row values. There are 27 possible permutations, which I can represent as 27 unique numbers between 0 and 222 by using a formula like =111+100*SIGN(N5-N6)+10*SIGN(O5-O6)+SIGN(P5-P6) [using the references for your example, then copy/paste/fill into a parallel block of cells].

    This scheme reduces your M2.1,S14 example to the number 220. I then scan the block of cells to see if you have consistently highlighted other 220 patterns, and it doesn't take me long to see a 220 pattern that you have also marked (M1.1,S15), and another 220 pattern that you did not mark (M11.1,S13). Obviously, I am not fully understanding the pattern detection logic you want to use, but perhaps it can still suggest a strategy you might use. Find a way to reduce each possible pattern in your detection scheme to a unique number or text string, then build a parallel block of cells that computes those values and you can analyze that parallel block.
    Quote Originally Posted by shg
    Mathematics is the native language of the natural world. Just trying to become literate.

  3. #3
    Registered User
    Join Date
    02-12-2023
    Location
    Praha
    MS-Off Ver
    2013
    Posts
    12

    Re: Problem: Finding patterns in a table

    Dear MrSHORTY,
    Thanks you for your message and your help. I wanted to take a moment to express my sincere gratitude for your recent efforts in trying to help me resolve my problem. Your willingness to lend a hand and provide guidance meant a lot to me.
    Although the issue isn't completely resolved, I want you to know that your support made a significant impact. Your new idea helped me gain a new perspective, and I truly appreciate the time and effort you dedicated to assisting me. It's reassuring to know that there are people like you who are willing to go the extra mile to help others. Once again, thank you for your kind assistance. I am truly grateful for your support, and it means a great deal to me.
    With heartfelt gratitude,
    Mark

  4. #4
    Registered User
    Join Date
    02-12-2023
    Location
    Praha
    MS-Off Ver
    2013
    Posts
    12

    Re: Problem: Finding patterns in a table

    Back to the topic and problem:
    Dear MrSHORTY your proposal is GREAT. Unfortunately, in practice, not everything goes right. I will try to explain my problem in more detail. Maybe I didn't understand everything or I'm delusional.
    I will use a fragment of the table (photo)
    1. I am looking for values in cells marked red,
    2. In some of the data, I saw some repeated patterns of temperature increase or decrease next to the values I was looking for
    3. I have marked the examples in the table
    4. Typical systems of increasing or decreasing temperature are e.g.
    Example 1
    The temperature change between the measuring points M3.1 and M2.1 in column S14 corresponds to a temperature increase by 0/4 (red) degrees [°C]. This increase is accompanied by a sharp temperature drop 7/0 (column S15) and a small temperature increase 0/1 (column S13) in the adjacent cells.
    Example 2
    a) searched increase 0/1 (M5.1/S15) (red)
    b) decrease 5/2 (M6.1-M5.1/S14)
    c) increase 0/2 (M5.1/S16)
    5. In some cases it is difficult to find a "pattern", but these cases are few.
    6. Due to the specificity of the process, there are time delays, i.e. the sought value occurs at the next measurement point. In my opinion, this is the main problem.
    7. I have no influence on the course of the process.

    Proposed solution (Dear MrSHORTY)
    Create the formula: Construct a formula that converts the pattern into a unique number or text string. Based on provided formula:
    1) Assign the base number: Start with a base number that represents the first permutation e.g. 111.
    2) Compare values and calculate offsets: Use the SIGN function to compare the values of the cells in the top row (N5, O5, P5) with the corresponding cells in the bottom row (N6, O6, P6). The result of the SIGN function will be -1 if the top row value is smaller, 0 if they are the same, and 1 if the top row value is larger.
    3) Multiply the offsets: Multiply the offset values by 100, 10, and 1, respectively, to create the appropriate weight for each comparison. This ensures that each comparison contributes to a unique number representation.
    4) Add up the results: Add up the base number and the weighted offsets to get the final unique number representation.

    MrShorta's idea works well, that is, the average effectiveness is around 40%. Can it be done differently? How can time shifts be included in the formula?

  5. #5
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,829

    Re: Problem: Finding patterns in a table

    After a few days, I will venture another response.

    "Can it be done differently?" -- Short but useless answer, Yes, it can be done differently. I am unable to figure out the logic you are using, though, so I cannot offer any improvements.
    "Can time shifts be included?" -- Again, yes, I'm sure they can be included, but I do not understand how you want to include time shifts so I cannot recommend anything.

    If it helps, the strategy I proposed basically boils down to representing each possible pattern as a unique base 3 number. Your description of your examples is focused on trios of cells where each member (digit) of the trio can take one of 3 values -- "decrease", "no change", "increase." Thus, my proposal assigns each pattern a unique base 3 number from 0 to 222.

    Some of your description suggests that size of change is part of this? Could we need base 5 numbers where each "digit" can take one of 5 values -- "large decrease," "small decrease," "no change," "small increase," "large increase" (resulting in base 5 numbers from 0 to 444)?

    Basically, this strategy will require you to identify how many "digits" (cells, m) you want to consider in each pattern group, and how many possible "values" (conditions, n) each digit can have. Then your pattern detection algorithm allows you to reduce your raw data into a table of m-digit base-n numbers. Then search the "reduced" table for the m-digit base-n numbers that represent the patterns you are wanting to highlight. In this way, this strategy is very flexible, and you can implement a variety of conditions in your pattern detection. But it will require you as the programmer to clearly and carefully define what kinds of patterns you want to detect. At present, I do not understand what kinds of patterns you are trying to detect, so I cannot offer anything more specific.

  6. #6
    Registered User
    Join Date
    02-12-2023
    Location
    Praha
    MS-Off Ver
    2013
    Posts
    12

    Re: Problem: Finding patterns in a table

    Dear MrSHORTY,
    Thank you very much for your answer. This time I am very sorry for my late reply. I think you are right. My expectations were too high. Mathematics is concrete. A few days ago, I checked my data with neural networks. Neural networks detected dependencies. I am not under any illusion. Probably at this stage it will be necessary to develop a database of patterns and test data. I will write about the progress. Thank you again for your help and your commitment. Your help turned out to be very valuable.
    With heartfelt gratitude,
    Mark

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. finding recurrent patterns
    By Kschott in forum Excel Programming / VBA / Macros
    Replies: 9
    Last Post: 03-24-2023, 04:25 AM
  2. Finding patterns in a database
    By neutral5 in forum Excel General
    Replies: 3
    Last Post: 11-30-2019, 04:05 PM
  3. Finding patterns in column
    By Jamesmc01 in forum Excel Formulas & Functions
    Replies: 2
    Last Post: 08-20-2019, 11:42 AM
  4. help finding patterns in columns?
    By HMexcel in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 02-05-2019, 01:54 PM
  5. [SOLVED] finding patterns in a range of cells
    By shart2k12 in forum Excel Programming / VBA / Macros
    Replies: 14
    Last Post: 12-27-2013, 11:07 AM
  6. Finding patterns
    By GaidenFocus in forum Excel Formulas & Functions
    Replies: 0
    Last Post: 12-06-2012, 03:59 PM
  7. Finding same patterns in groupings of cells
    By modoud in forum Excel Formulas & Functions
    Replies: 1
    Last Post: 07-19-2007, 01:19 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1