+ Reply to Thread
Results 1 to 13 of 13

How to organize and filter data with multiple categories and subcategories

  1. #1
    Registered User
    Join Date
    01-17-2018
    Location
    Washington, D.C.
    MS-Off Ver
    2016
    Posts
    6

    How to organize and filter data with multiple categories and subcategories

    Context: I'm creating a database of resources for my team to use. The goal of the database is to allow people to sort the resources across multiple dimensions: topic, specificity, which projects they're related to, etc. All the data is text - there are no numbers.

    Problem: Many resources belong to multiple categories within one dimension. For example, a specific resource might pertain to more than one topic. Ideally I'd like to be able to enter multiple topics into one cell separated by commas (for topic it'd be "Cyber, Defense, Innovation) and then be able to sort by any of the individual values in the cell. That doesn't seem to be possible though.

    Tl;dr: how do you filter text data that varies across multiple categories, within which there are sub-categories? And how do you efficiently organize that data in your sheet so it can be filtered?

    Thank you so much for any help you can provide! I really appreciate it.

    Cheers!

  2. #2
    Forum Expert
    Join Date
    11-22-2016
    Location
    Cornwall,UK
    MS-Off Ver
    office 365
    Posts
    4,240

    Re: How to organize and filter data with multiple categories and subcategories

    Welcome to the forum
    Access is generally a better tool than Excel for this type of thing - but Excel is perfectly adequate for smaller databases
    It sounds like you are wanting to use Excel? Is that correct?

    To avoid anyone having to re-rcreate your data etc, please attach a workbook to get things moving
    - illustrate your needs with clear examples of what you are trying to do
    - your examples should comprise the more complex query types
    - make text anonymous but still typical of your data

    Anyone helping you will need to understand
    - how the data looks
    - how it interrelates
    - how complex queries may be
    - how sorting should work
    - how the output should look
    etc

    Is it your intention that the database holds all the information or acts more as an index (with hyperlinks etc) to other files\intranet pages?

    Where will the database reside?
    - local server?
    - SharePoint or similar?
    (there may be different VBA functionality issues to consider)

    Estimate of ultimate dimensions of database?
    - how many categories\sub-categories\rows of data etc
    - after initial set-up, how frequently will database be updated?

    To attach a workbook:
    - click on GoAdvanced
    - look below and click on ManageAttachments
    - follow screen instructions

    I am asking these questions to kick this off, not because I can necessarily provide you with a solution - that depends on how complicated it all looks
    Last edited by kev_; 01-20-2018 at 04:47 PM.
    Click *Add Reputation to thank those who helped you. Ask if anything is not clear

  3. #3
    Registered User
    Join Date
    01-17-2018
    Location
    Washington, D.C.
    MS-Off Ver
    2016
    Posts
    6

    Re: How to organize and filter data with multiple categories and subcategories

    Hi Kev, thanks for answering! I've tried to provide all the necessary info below, and I attached the spreadsheet.

    Each of the rows is a reference to a useful resource for new members on our team. Each has a link to the internet or our Sharepoint site, and the other columns contain information about the categories the resources fall under. A given resource can relate to multiple subjects, regions, and projects, and the resources are also ranked by difficulty. That totals four dimensions, and we might add a couple more later, but it won't get excessive.

    It'd be easy enough to use slicers and the filter feature if the resources only belonged to one category (e.g., cybersecurity or Europe) on each dimension (e.g., subject or region), but they don't. Lots of them belong to multiple categories under the same dimension. For instance, a paper on Russian cyberwarfare could be tagged with both cybersecurity and geopolitics for subject, or with both Europe and FSU for region. We'll also be adding more categories as the sheet expands. Eventually there'll be dozens of categories and hundreds of rows.

    The goal is to have an unskilled user open the spreadsheet, check the boxes (or similar) on all the categories they're looking for (or use slicers, filters, etc), and have the references filtered by those categories. Right now I can use slicers and the filter function, but I can't assign multiple categories under one dimension to any reference - I just have to pick the best one for each, which isn't ideal.

    The spreadsheet will live in Sharepoint, and it will frequently be updated by multiple users, not all of whom will understand how the spreadsheet works.


    Let me know if any of this isn't clear! I've never had to explain an Excel project in words before to someone not standing next to me, so I'm happy to elaborate further! Again, thanks so much for any help you can provide.
    Attached Files Attached Files

  4. #4
    Forum Expert
    Join Date
    11-22-2016
    Location
    Cornwall,UK
    MS-Off Ver
    office 365
    Posts
    4,240

    Re: How to organize and filter data with multiple categories and subcategories

    For some reason the antivirus program on my Laptop is screaming "KeepOut" of the forum - so I am staying away until that has been resolved. It is unlikely to be a serious issue but better safe than sorry. I use McAfee - so if it is screaming at me, it will be screaming at lots of others too! I am currently accessing via a specific tablet which is not connected to anything important! (Just in case )

    I have looked at your reply and workbook - and your request looks totally do-able

    I will test my ideas tomorrow:
    - the values by line for Subject, Region, Level and Project to be held in 4 columns (one each)
    - with each cell value being comma delimited (exactly what you requested)
    (ie Project column value = A,B,C,D if all projects relevant instead of current "Yes" in 4 columns)
    - a cell can hold over 30,000 characters which is more than sufficient for subject and region categories

    Data Filter cannot handle your requirements, but I expect Advanced Filter to cope with everything you want
    - filtering the values is straightforward
    - BUT sorting them will require a specific VBA procedure
    - filtering selection can be fronted in a way similar to slicers for your users

    It would be better if my test database started looking as similar as possible to what you ultimately want, especially the 4 filter columns
    - (rather than me create something random) could you reply with a small list of valid "subjects" (20 to 30)
    - the other columns I am ok with

    I aim to keep VBA to a minimum, and make Excel's basic functionality do as much as possible
    - Excel is faster than VBA
    - there are a few VBA issues with Sharepoint
    - there are also some Excel issues with Sharepoint to bear in mind

    Let's get the filtering nailed first before moving onto sorting the output

    If you want to test the comma delimited idea, you can do that (in a limited way) with Data Filter
    - create a column with some text values that are comma delimited
    - click on the fitter arrow
    - select Text Filter
    - select Text Contains
    - type a (different) value equal to one of the delimited values in each box
    - the second condition = "contains" (it's the last item in dropdown list)
    AND = both conditions satisfied for item
    OR = either condition satisfied for item
    - Data Filter can handle 2 conditions in one column
    - Advanced Filter is much more impressive in what it can handle
    Last edited by kev_; 01-23-2018 at 09:58 AM.

  5. #5
    Forum Expert
    Join Date
    11-22-2016
    Location
    Cornwall,UK
    MS-Off Ver
    office 365
    Posts
    4,240

    Re: How to organize and filter data with multiple categories and subcategories

    This is deliberately unsophisticated
    - We need to be sure that advanced filter can give you the results you need
    - I have made no attempt at this stage to make it easy for the user (that will not be an issue later)
    - sorting has also been ignored until later

    Advanced filter alows the user to be sophisticated with filters but Excel requires the filter criteria tabulated a specific way., and is fussy with search strings etc.
    Everything in the attached is raw to reveal the nuts and bolts.

    How it works:
    In sheet "Criteria" :
    - a usable search string is formula generated in row 4 (based on rows 2 & 3 values)
    - the operator can be selected separately for each column
    - "Add to Criteria" copies the data below
    - several crieria rows can be accumulated into the single data filter
    - each criteria row is independent of other rows
    (ie the results = everything on row 8 PLUS everything on row 9 etc)
    - "Filter Data" takes you to the filtered data

    To test:
    - have a look at the data first and then (in sheet "Criteria") select appropriate values from dropdowns
    - click "Add to Criteria" when the whole row is what you want
    - everything in the row must be satisfied for something to appear in the results
    - to delete a criteria row(below row 7), select any cell in the row
    - click on "Filter Data" to see your results (in sheet Data)
    - to show all data again on sheet "data" take menu item Data\Clear
    - cell B3 allows user to enter freeform text or select from dropdown

    Let me know how you get on

    (In case you are puzzled...) the items that appear to be in A8 and A9 beow are notes I added on to the image

    AdvancedFilter.jpg
    Attached Files Attached Files
    Last edited by kev_; 01-25-2018 at 04:19 AM.

  6. #6
    Registered User
    Join Date
    01-17-2018
    Location
    Washington, D.C.
    MS-Off Ver
    2016
    Posts
    6

    Re: How to organize and filter data with multiple categories and subcategories

    This is FANTASTIC! It's exactly what I need Thank you so much - I knew there had to be a way to do this!

    I played around with it to see what it does, and this already is really well suited to form the basis of the future resource spreadsheet I'll be using. The categories you chose are fine too - good that you came up with so many. I'd also be really curious to learn how you used the advanced filter tool and what VBAs you used.

    How would you go about improving user friendliness and introducing a sort feature?

    Also, sorry for the few days' lag time in responding to your last post (things got busy). I'm moving ahead now with our knowledge management system, so my future response times will be much quicker!

  7. #7
    Forum Expert
    Join Date
    11-22-2016
    Location
    Cornwall,UK
    MS-Off Ver
    office 365
    Posts
    4,240

    Re: How to organize and filter data with multiple categories and subcategories

    The delay was not an issue - I can still remember where I was heading with your database! I am pleased that you can see this method working well for you.

    We need to take account of how the database is going to be maintained as well as searched
    - consistency of approach in "tagging" is the key to better quality search results (cybersecurity may not be found if user is searching for cyber-security etc)
    - we can do something to alleviate this (search for both even if user only enters 1)
    - data maintenance input tools need to mirror data searching tools

    It would be best to try allow for your future plans during testing
    - 2 more dimensions will be added (I will invent something)
    - these can then be switched off until you are ready to use them
    - are you considering adding a "year" field to allow user to exclude outdated resorces?

    It is crucial that you test everything on Sharepoint as we proceed
    - Sharepoint is a great tool but a few Excel features do not work and any such issues arising are best identified sooner rather than later .

    It could be useful to line up a user to do some quick ongoing "usability" testing with you
    - it is easy to miss something when too closely involved in the "build"
    - the user quickly notices if something is not intuitive etc

    What is the earliest version of Excel to allow for?

    I aim to have an updated version posted within the next 48 hours -more user firiendly and with basic sorting capabilities added
    We'll need to bounce the test file back and forth for a few days, trying out a few different ideas etc. There are a several options available - I would be astounded if we get it right first time!
    Last edited by kev_; 01-28-2018 at 09:55 PM.

  8. #8
    Registered User
    Join Date
    01-17-2018
    Location
    Washington, D.C.
    MS-Off Ver
    2016
    Posts
    6

    Re: How to organize and filter data with multiple categories and subcategories

    That's a very good point on consistency of tagging. It would be great to have a feature that would allow for variants (cybersecurity vs cyber-security), but we don't necessarily have to include it – I'll have the chance to instruct all the new users of the sheet how to enter new data (i.e. be 100% consistent in tag use; list tags in alphabetical order separated by commas, etc.).

    Excellent point about adding a couple more categories too – always good to plan for unexpected additions in advance. The year field is a very good idea, especially since cybersecurity (and any resources relating to it) changes dramatically year on year.

    I'll start testing the usability and functionality of the sheet on Sharepoint with a colleague.

    Fortunately everyone using it will be on Excel 2016, so that shouldn't be an issue!

  9. #9
    Forum Expert
    Join Date
    11-22-2016
    Location
    Cornwall,UK
    MS-Off Ver
    office 365
    Posts
    4,240

    Re: How to organize and filter data with multiple categories and subcategories

    It should be easy to design a simple input form to make life easy for the user
    - and use data validation to control the user's use of only only authorised tags
    (Only "trusted" individuals should be authorised to create new tags - they will be key players!)
    - VBA can sort the tags alphabetically and put the commas in there
    I would not trust a user with inserting commas or putting words in sequence
    - half of them probably can't remember the alphabet
    - and the others are guaranteed to land you with you with a bagatelle of spaces, dashes, colons, full stops, commas etc
    Searching variants can be handled - should only really arise if searching with free text
    - eg searching for "cyber" rather than tags "cybersecurity" or "cyberfraud"
    - tags should be unique - if someone sets up 2 tags that are too similar then can use VBA to remove one of them and convert the data to the chosen tag (problem removed for ever)

  10. #10
    Forum Expert
    Join Date
    11-22-2016
    Location
    Cornwall,UK
    MS-Off Ver
    office 365
    Posts
    4,240

    Re: How to organize and filter data with multiple categories and subcategories

    I have been rather busy today, and so have had little time to spend on your thread
    Attached workbook includes a potential user-friendly dimension selector for you to consider

    - click on cell once and it turns green (= I want this)
    - click on cell again and it turns red (= exclude this)
    - click on cell again to remove color
    - to clear all selected items in a column click on blue header cell

    VBA is behind the sheet and is simple
    - it comprises one short Selection Change Event macro
    - here are no complicated buttons or checkboxes to maintain

    It is not hooked into the database at this stage, but a click of a button would drive creation of filter criteria based on user selections

    Any thoughts?
    Attached Files Attached Files

  11. #11
    Registered User
    Join Date
    01-17-2018
    Location
    Washington, D.C.
    MS-Off Ver
    2016
    Posts
    6

    Re: How to organize and filter data with multiple categories and subcategories

    Re: your second to last post - I really like those long-term permanent fixes you proposed for the input. Always good to make things idiot-proof

    The selection option is great - it's intuitive and user-friendly. Looking forward to seeing what it'll look like when plugged into the database!

  12. #12
    Forum Expert
    Join Date
    11-22-2016
    Location
    Cornwall,UK
    MS-Off Ver
    office 365
    Posts
    4,240

    Re: How to organize and filter data with multiple categories and subcategories

    Updating the Database (see attached workbook)
    - demonstrates comma separators being introduced automatically
    - user forced to use approved tags only
    - some columns restrict user to only one tag

    I wanted you to have a look at how database tags could be updated (similar to yesterday's search)
    - it needs a bit of tidying up (see below) - but gives a good indication of how things might work
    - not fully tested so you may spot something I haven't

    To amend current tag values in Sheet "Data":
    - click any cell (except columns A,B & I)
    - takes you automatically to Sheet "Lists"
    - the current tags against selected document are highlighted in green
    - (as before) clicking on a cell toggles the cell colour
    - (as before) clicking on the blue header cell allows user to clear all tags in that columns (after message pop-up)
    - clicking back on "Data" without clicking on "Update Tag Values" aborts any changes made
    - clicking on "Update Tag Values" overwrites values in "Data"

    To add another "paper" to sheet data
    - type the name into cell in next row in column A
    - ignore column B for the moment
    - select cell in column C and everything is as above

    Tidying up: (tomorrow!)
    - the VBA would be simplified if the column matching in both sheets was built into the workbook
    (lookups are currently done via VBA - I have changed my mind!!)
    - cell Q1 in sheet "Lists" holds the row number of the item currently being edited
    - probably better to add a hidden worksheet to hold some values that VBA can simply refer to
    - the user may find it useful to see both the "end result" and what was there originally before clicking "Update Tag Values" (see question below)
    - procedures ViewCurrentTags and Update tags share overlapping requirements which are currently duplicated in the code
    - I spotted one minor thing that the code is doing wrongly when identifying tag matches (let's see if you spot it too!!)
    - column I is not integrated into anything just yet (see question below)

    Questions
    - HOW ARE YOUR DOCUMENTS NUMBERED? Do you want to autonumber?
    - rather than change cell colour (which loses original status) would it be better to use strikethrough font for proposed deletions and red font(?) for proposed additions?
    - Column I is another one of my random thoughts(??) - would it be useful to have a field where users could include a reference to other documents that are linked in some way?

    - are we still heading the the correct direction?
    Attached Files Attached Files
    Last edited by kev_; 01-31-2018 at 03:57 PM.

  13. #13
    Registered User
    Join Date
    01-17-2018
    Location
    Washington, D.C.
    MS-Off Ver
    2016
    Posts
    6

    Re: How to organize and filter data with multiple categories and subcategories

    - Auto-numbering would work well
    - For the point about strikethrough and red font, are you referring to allowing the user to see the "end result" compared to the original result? I'd be curious to see what this would look like - all the changes you've envisioned so far have been spot on.
    - Column I would be useful to have in reserve, maybe for use at a later date, but at this point I think it'd be too much to ask of users to add additional references for all or most of the sources that they input.
    - It'd also be useful to have a "select all" feature on the lists page so the user can quickly highlight all the tags in green

    I also noticed that it's not possible to deselect all tags except for Project - if I try to do so and hit "update tag values," it just reuses the tag value that was previously selected.

    How do you search for an specific set of tags? Right now it looks like it lists any document that matches at least one of the selected tags, but I haven't found a way to search for documents that match all categories in a set of tags. Same thing with an exclusionary search - it'd be useful to be able to exclude a number of tags from the search (right now I think the <> operator returns values that exclude at least one - but not all of - the tags).

    I can feel us getting close to the final product!

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 5
    Last Post: 02-07-2014, 05:07 AM
  2. Macro to filter and sort by multiple categories & Send to correct sheet
    By lax2734 in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 07-08-2013, 08:35 PM
  3. Organize data that has the same name from multiple sheets into one
    By babypooh in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 11-01-2012, 08:58 PM
  4. Replies: 2
    Last Post: 07-17-2012, 03:05 PM
  5. create a macro to filter categories and copy data
    By gryffin13 in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 07-02-2012, 08:20 AM
  6. Multiple Data in One Column - How to Organize?
    By Peeekay in forum Excel General
    Replies: 7
    Last Post: 02-01-2011, 05:45 PM
  7. [SOLVED] Switching Subcategories into Categories and Vice Versa
    By [email protected] in forum Excel Formulas & Functions
    Replies: 7
    Last Post: 04-17-2006, 10:30 AM
  8. Macros to Organize the Data from multiple files to one file
    By TEAM in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 07-19-2005, 05:05 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1