+ Reply to Thread
Results 1 to 14 of 14

options to do multiple linear regression with constraints (coefficients >= 0 and sum=1)

  1. #1
    Registered User
    Join Date
    01-23-2017
    Location
    Pune, India
    MS-Off Ver
    2010
    Posts
    5

    options to do multiple linear regression with constraints (coefficients >= 0 and sum=1)

    I want to run a multiple regression in EXCEL with constraints (all coefficients >= 0 and sum of all coefficients =1). Want to find the best fit regression line equation with the mentioned constraints.
    I would like to know different options to achieve my objective and which one would be quick and easy.

    I have around 230 rows of data with 10 independent variables.
    I am attaching data file.

    I am new to EXCEL.
    Things I am contemplating about are: Using LineST function, using solver etc.

    Is it possible to club the constraints with LineST function?
    Is it possible to use Solver for achieving my objective?
    Can I do it through macro using LineST?

    Please guide. Your help is very much appreciated.

    Note: No constraints on intercept in the regression equation.
    Attached Files Attached Files

  2. #2
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,819

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    There are probably multiple possible approaches. My first inclination, with those particular constraints is to use Solver. Being new to Excel, are you familiar with Solver? Here's how I would approach this:

    1) Enter your best first guess values for the coefficients of the regression equation y=sum(Ai*xi)+B. You might try 0.1 for each, or use the LINEST() function to get these first estimates, or some other reasonable first guess. I put them in P1:Y1, with B in Z1
    2) Add a column to compute yequation from the coefficients and the x's. This could be involve a SUMPRODUCT() function or whatever. Be sure to pay attention to relative and absolute references, to make this easy to copy. I used =$Z$1+sumproduct(B2:K2,$P$1:$Y$1) in M2 and copied down. (Help files for functions: https://support.office.com/en-us/art...6-c6d90033e188 )
    3) Compute your desired objective function. For least squares, this might be =SUMXMY2(M2:M229,A2:A229) or whatever you deem the most appropriate obective.
    4) In order to deal with the "sum(Ai)=1" constraint, I added a cell =SUM(P1:Y1) to compute the sum of the coefficients.
    5) Call Solver and tell it to
    5a) Set Target Cell: cell containing the SUMXMY2() or other objective function.
    5b) To a minimum (or maximum or value as appropriate to the chosen OF).
    5c) By changing: P1:Z1
    5d) Subject to constraints that P1:Y1>=0 and cell with SUM(P1:Y1) function=1.

    In trying a few different starting values, I noted that Solver converged on different solutions, depending on what I chose for the starting value. These kind of problems can frequently have multiple possible solutions, and you may need to further explore the solution space to see how to identify the absolute best fit according to those constraints.
    Quote Originally Posted by shg
    Mathematics is the native language of the natural world. Just trying to become literate.

  3. #3
    Forum Expert dflak's Avatar
    Join Date
    11-24-2015
    Location
    North Carolina
    MS-Off Ver
    365
    Posts
    7,920

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    Hey Mr.Shorty. While you were posting I was doing it. . I think we cover the same ground, but I suppose two views are better than one, so here goes:

    Digging this one up from when Professor Newton taught me math.

    The least squares fit method depends upon minimizing the sum of the squares of the difference between a computed value and its actual value.

    The Add-In solver can do this. I’ve used it on complex equations like A*SIN(B*t+C) + D*Ln(t) + E. Try solving that algebraically. This is an equation for growth adjusted for seasonality.

    First, go to options and enable the solver Add-in. I suggest that you take it out when not using it as it seems to slow down Excel from launching and may also crash it at times. At least in my at work copy of Excel 2013.

    The key to using solver is to set up the problem properly. I did the setup on Sheet 2. In Cells I1 and I2 I set up the coefficients M (slope) and B (intercept) and populated them with an initial guess of 0.5 each.

    I set cell I3 equal to the sum of I1 and I2.

    In column C I set up a formula to compute Y using these coefficients =$I$1*B2+$I$2.

    Column D contains the difference between observed and computed: =A2-C2.

    Column E contains the square of the difference: =D2^2

    Cell J6 contains the sum of the square of the differences.

    Now we can set up solver (see picture).

    What we want to do is minimize the value of Cell J6 by changing the values in Cells I1:I2. With the constraints that I1 and I2 are non-negative and that the value in Cell I3 is 1.

    Click on solve and solver will go to work.

    I did this for the first set, and it came out with a slope of 1 and an intercept of zero. It does seem like a trivial solution.

    I copied and pasted some of the other X ranges in and they also came out 1 and zero. There is a difference in the slope in the 9th decimal place.
    Attached Images Attached Images
    Attached Files Attached Files
    One spreadsheet to rule them all. One spreadsheet to find them. One spreadsheet to bring them all and at corporate, bind them.

    A picture is worth a thousand words, but a sample spreadsheet is more likely to be worked on.

  4. #4
    Registered User
    Join Date
    01-23-2017
    Location
    Pune, India
    MS-Off Ver
    2010
    Posts
    5

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    Thank You so much MrShortly for steps and to Dflak for the screenshot and small version built for me. With your help, I successfully achieved my objective using solver.

    Now I want to automate it i.e. want this to happen with macro where data range can be given and then macro does the job without any further manual intervention.
    Macro with Solver (if feasible)is fine option for me. Can any of you please help / guide?

    Also, Can this automation be achieved by any other means apart from using macro-solver?
    Thanks in advance.

  5. #5
    Forum Expert dflak's Avatar
    Join Date
    11-24-2015
    Location
    North Carolina
    MS-Off Ver
    365
    Posts
    7,920

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    Yes, it's possible to do this via VBA. I'll have to record a macro for running Solver.

    I have some questions that might be germane: will you always have 229 rows of data (not a big deal) and will you always have 10 X-values (again, not a big deal). It will tell me how flexible I have to make the code.

    I'll report the answers in a table Xn | M | B. So you'll have a list from 1 to 10 (or whatever) with the associated M and B.

    I have some other projects in front of this one, so I may not get to it until tomorrow.

  6. #6
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,819

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    A few examples, while dflak prepares his example:

    Some basics about using Solver in VBA: http://peltiertech.com/Excel/SolverVBA.html
    Here's one with Solver in a loop: http://www.excelforum.com/showthread.php?t=1166949
    Here's on from stackoverflow showing the basic structure of a change event procedure: http://stackoverflow.com/questions/1...e-sheet-change Once you have the basic Solver code, you can place it inside of a change event procedure.
    I would be more inclined to use a calculate event procedure. This one shows Goal Seek called from a calculate event, replacing the goal seek stuff with Solver stuff should be straightforward: http://excel.kingofmath.com/?p=267

  7. #7
    Forum Expert dflak's Avatar
    Join Date
    11-24-2015
    Location
    North Carolina
    MS-Off Ver
    365
    Posts
    7,920

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    I had to look up a couple of things and that's why I participate in the forum; it forces me to learn stuff.

    Here it is.
    Attached Files Attached Files

  8. #8
    Registered User
    Join Date
    01-23-2017
    Location
    Pune, India
    MS-Off Ver
    2010
    Posts
    5

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    Thank you very much dflak for your short demo file and macro code. Taking it as guideline I have now developed my 1st ever macro and that too catering some good level problem. :-)
    Thanks mrshortly for the links. It will be really great if I can get any other link / tutorial which demonstrates effective usage of solver in a macro with loop so that I can handle multiple solver runs (each run with a slightly little different criteria say input cells).

    My current problem is, I am getting 'Solver Results' window every time Solver has been run. Because my code requires multiple solver runs, I need to click on 'OK' manually every time so that macro could complete its task. Requesting your help on following questions.
    1. With what code in macro I can just STOP appearance of 'Solver Results' on the window screen? However in doing so, I still want the results to get pasted by solver in the intended cells of worksheet. If it is Not possible to stop the appearance, with what code in macro, I can achieve automatic click on 'ok' button of 'Solver Results' window? Please help.
    2. Is there any way I can increase no. of iterations solver considers (by default it is 5) before throwing out 'Solver Results' window showing convergence achieved mesg?
    Attached Files Attached Files

  9. #9
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,819

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    1. With what code in macro I can just STOP appearance of 'Solver Results' on the window screen? However in doing so, I still want the results to get pasted by solver in the intended cells of worksheet. If it is Not possible to stop the appearance, with what code in macro, I can achieve automatic click on 'ok' button of 'Solver Results' window? Please help.
    As explained in the first link (about 5th "paragraph" down below the first sample recorded code):
    Quote Originally Posted by Jon Peltier
    SolverSolve has an optional UserFinish argument; if UserFinish is False or omitted, the second dialog shown above will ask the user whether to save the optimization, but if UserFinish is True, Solver will end without the dialog.
    2. Is there any way I can increase no. of iterations solver considers (by default it is 5) before throwing out 'Solver Results' window showing convergence achieved mesg?
    I'm not sure I understand your question. The message you posted suggests trying a smaller convergence setting. What is your current convergence setting, and have you tried a smaller setting? What is the deeper concern here?

  10. #10
    Registered User
    Join Date
    01-23-2017
    Location
    Pune, India
    MS-Off Ver
    2010
    Posts
    5

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    Thank You mrshortly. I somehow overlooked it however now with your help it is working. Thanks a ton.

    Now I think I am facing with possibly last but crucial query on my 1st macro :-)
    Inside the loop for solver, I need to change the rows (removing 1 old row and adding 1 new row of the data series) of the formula in a cell on which Objective Function is based on.

    For example:
    Objective function is Minimise $V$2.
    For 1st loop I want Formula in V2 as SUM(S2:S37) where as for 2nd loop of solver I want formula in V2 to be SUM(S3:S38).

    I am using recording(for solver) + coding(for looping) approach. Macro I recorded has a Formula in code as "=SUM(R[0]C[-3]:R[35]C[-3])" i.e. pointing to S2:S37 (from V2)

    To serve my purpose of changing rows (used for SUM in V2) for each loop iteration, I want to make this SUM formula 'variable based' say for example

    Range("V2").Select
    ActiveCell.FormulaR1C1 = "=SUM(R[n1]C[-3]:R[p1]C[-3])"

    where n1 and p1 are variables used to facilitate change of rows so that my purpose is served. However problem is macro NOT allowing variable inside the said formula and gives error.
    How can I achieve my purpose? Please guide.

    Please refer attachemnt for screenshots. Thanks much in advance
    Attached Files Attached Files

  11. #11
    Registered User
    Join Date
    01-23-2017
    Location
    Pune, India
    MS-Off Ver
    2010
    Posts
    5

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    I could figure out the solution as :
    Range("V2").Select
    ActiveCell.FormulaR1C1 = "=SUM(R[" & n1 & "]C[-3]:R[" & p1 & "]C[-3])"

    and it is serving my purpose. Thank You.

    Now, I need a similar thing (relative and dynamic reference) for the cells mentioned in Constraints of Solver. For example see highlighted part:
    SolverAdd CellRef:="$Y$2:$AH$2", Relation:=3, FormulaText:="0"

    In the code above, instead of putting in as column AH, I would like to use it with reference to a user input (captured from a cell in worksheet say V5).
    Say if this user input (V5) is 10, I would like the EXCEL to choose a column which is 10 places from 'Y' i.e. AH
    Say if this user input (V5) is 15, I would like the EXCEL to choose a column which is 15 places from 'Y' i.e. AM

    My requirement is: Depending on V5 value (allowed range is 1 to 15), Solver constraint should change accordingly.

    mrshortly / dflak, Can you please help ? Thank You in advance.

  12. #12
    Forum Expert Alf's Avatar
    Join Date
    03-13-2004
    Location
    Gothenburg/Mullsjoe, Sweden
    MS-Off Ver
    Excel 2019 and not sure I like it
    Posts
    4,758

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    (V5) is 10, I would like the EXCEL to choose a column which is 10 places from 'Y' i.e. AH
    As "Y" is the 25th column, the column that is 10 places from "Y" is then the 35th column i.e. column "AI", the "AH" I would say is 9 places from "Y".

    Keeping this in mind you could probably use something like

    Please Login or Register  to view this content.
    be aware that the cell V5 must be on the same sheet as your solver model.

    Alf

  13. #13
    Registered User
    Join Date
    02-15-2019
    Location
    Oslo, Norway
    MS-Off Ver
    2013
    Posts
    3

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    Hi, I'm new to this site and I'm working on a similar problem to the one on this thread. I'm having trouble creating a solver loop in VBA. Should I post a new thread or as a reply to this thread?

    Thanks for the help!


    Regards

    B

  14. #14
    Forum Expert dflak's Avatar
    Join Date
    11-24-2015
    Location
    North Carolina
    MS-Off Ver
    365
    Posts
    7,920

    Re: options to do multiple linear regression with constraints (coefficients >= 0 and sum=1

    It is always a good idea to post your own thread. Copy and paste a link back to this thread if you think it will help. Also include a sample workbook with representative data and if it is appropriate, mock up a sheet showing what you want the results to look like.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 1
    Last Post: 10-07-2015, 02:55 PM
  2. Multiple Linear Regression
    By BKB2003 in forum Excel Charting & Pivots
    Replies: 1
    Last Post: 11-11-2014, 10:03 AM
  3. Replies: 4
    Last Post: 02-10-2014, 04:37 AM
  4. how to constrain coefficients in a multiple regression using solver
    By psheeha1 in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 08-27-2012, 11:02 PM
  5. Multiple regression coefficients are not seen
    By Dr.Appalayya in forum Excel Formulas & Functions
    Replies: 4
    Last Post: 11-15-2009, 04:54 AM
  6. Data Analysis Add In - Storing Multiple Regression Coefficients
    By kilmacanty in forum Excel Formulas & Functions
    Replies: 2
    Last Post: 11-05-2008, 01:11 PM
  7. how do i graph multiple linear regression?
    By enortirol in forum Excel General
    Replies: 0
    Last Post: 02-21-2006, 09:25 AM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1