| |
| ece6554:project_invcartpend [2022/04/23 17:08] – [Report Deliverable Considerations] pvela | ece6554:project_invcartpend [2024/08/20 21:38] (current) – external edit 127.0.0.1 |
|---|
| My units might be off for the friction, but the values are fine. | My units might be off for the friction, but the values are fine. |
| |
| | ===== Implementation ===== |
| | |
| | Functional code stubs for the implementation are provided in the {{ ECE6554:projects:invpendcart.zip | invpendcart zipfile}}. They implement a constant control signal that most definitely fails to do the job, but provide enough structure to complete the project. Missing is a reasonable implementation of a single layer Gaussian radial basis function neural network. That's left to be coded up as a class and properly used as part of the Inverted Pendulum on a Cart neuro-adaptive controller. |
| ====== Activities ====== | ====== Activities ====== |
| ---------------------- | ---------------------- |
| Implement an adaptive system and compare the outcomes. Run the adaptive system twice. Once with the incorrect values, then once with the final adaptive coefficients from the first run as that initial adaptive coefficients for the second run. Compare the quantitative values of the incorrect run, the first adaptive run, and the second adaptive run (where compare means to provide the outcomes and to discuss them). | Implement an adaptive system and compare the outcomes. Run the adaptive system twice. Once with the incorrect values, then once with the final adaptive coefficients from the first run as that initial adaptive coefficients for the second run. Compare the quantitative values of the incorrect run, the first adaptive run, and the second adaptive run (where compare means to provide the outcomes and to discuss them). |
| |
| ===== Step 2: Neuro-Adaptive Controller ===== | ===== Step 2: Implement Single-Layer GRBF Neural Network for Function Approximation ===== |
| | |
| | To be fleshed out, but main idea is to first approximate a real function like $\sin(x)$ or $\mathrm{sinc}(x)$ or some polynomial function, over a fixed interval. The second is to approximate a function of two variables that can be a thin plate spline or some made up function of two variables with a non-trivial but not too complex surface plot. Once you get the hang of that, you should be able to implement for arbitrary dimensions, just that it gets a bit more complex. Then you'll be ready for the adaptive control version. |
| | |
| | For completeness, implement a neuro-adaptive system for a scalar system as noted in //Tip 1//. Follow through on the exploration of //Tip 2//. If this is your first time digging deep into function approximation with Matlab, then you'll probably have to check out //Tip 3//. |
| | ===== Step 3: Neuro-Adaptive Controller ===== |
| |
| The inverted cart-pendulum system is a bit tricky to work with from a nonlinear control perspective because the single control acts through the two states to "stabilize." Furthermore, as a nonlinear system, it is characterized by a relatively large region of attraction for the linear controller. I am not sure how large it is once the parameters vary, but for the correct parameters it can be pretty large (covering a $\pm 60^\circ$ deviation from up). The added value of a nonlinear controller might not be worth the effort depending on the domain of interest for this system. | The inverted cart-pendulum system is a bit tricky to work with from a nonlinear control perspective because the single control acts through the two states to "stabilize." Furthermore, as a nonlinear system, it is characterized by a relatively large region of attraction for the linear controller. I am not sure how large it is once the parameters vary, but for the correct parameters it can be pretty large (covering a $\pm 60^\circ$ deviation from up). The added value of a nonlinear controller might not be worth the effort depending on the domain of interest for this system. |
| How well it works is a function of how many basis elements you have, their bandwidth, etc. Make sure that they cover the domain of operation (including the extended area where you'd like to have stability). Note that the nonlinearities are independent of the cart position $x$, so you should not include it in the function approximator. That keeps the network size smaller by not having to learn any $x$-dependence. The input dimension is 3 instead of 4. | How well it works is a function of how many basis elements you have, their bandwidth, etc. Make sure that they cover the domain of operation (including the extended area where you'd like to have stability). Note that the nonlinearities are independent of the cart position $x$, so you should not include it in the function approximator. That keeps the network size smaller by not having to learn any $x$-dependence. The input dimension is 3 instead of 4. |
| |
| ==== Tip: Neuro-Adaptive Updates ==== | ===== Step 4: Matched and Unmatched Uncertainty ===== |
| |
| When implementing new things, it is important to start from a position of strength. Do not try to do it all in one step given that there are many smaller steps that you most likely have never done. Break the system down into smaller pieces. From the first step, you should already have functioning adaptive controllers on only the linear system, plus running on the true nonlinear dynamics (but with a linear reference model). Naturally, this means you also have the non-adaptive versions working, as those should have been an initial testing step prior to incorporating adaptation. | The inverted cart pendulum system has a set of nonlinearities that are matched, plus a set that are not. Consider adaptive control for that set: |
| | |
| | \begin{equation} |
| | \dot x = A x + f_u(x) + B(\theta) (u + f_m(x)) |
| | \end{equation} |
| | |
| | Instead of using a GRBF-NN, consider cancelling only the part that can be cancelled and leaving alone the part that can't be cancelled. Manipulate the equations of motion to get the form of $f_u(x)$ and or $f_m(x)$. What are the basis functions in $\Phi(\cdot)$? Come up with an adaptive controller for the unmatched uncertainty by considering it to be a non-vanishing disturbance and compensating for it appropriately. Implement it and perform the proper simulations and comparisons based on the earlier steps. |
| | If you are taking the Nonlinear Control class and have covered ISS, then you should be able to come up with a targeted adaptive controller similar to the disturbance rejection one that compensates for the state-dependent unmatched uncertainty with better boundedness or stabilization properties relative to naive adaptive disturbance compensation. |
| | |
| | |
| | **Note:** The uncancelled parts can actually be recovered using ideas from concurrent learning. Basically store the evolution of the system and use it to recover what that part should look like, assuming that the remainder of the adaptive controller was functioning properly. As data gets collected, one can perform standard regression on the data. It is even possible to try to correct for that nuisance term, but doing so is outside of the scope of this class. Just letting you know it is possible, not part of the actual step. |
| | |
| | |
| | ===== Implementation Tips ===== |
| | |
| | ==== Tip 1: Neuro-Adaptive Updates ==== |
| | |
| | When implementing new things, it is important to start from a position of strength. Do not try to do it all in one step given that there are many smaller steps that you most likely have never done. Break the system down into smaller pieces. From the first step, you should already have functioning adaptive controllers on only the linear system, plus running on the true nonlinear dynamics (but with a linear reference model). Naturally, this means you also have the non-adaptive versions working, as those should have been an initial testing step prior to incorporating adaptation. This is what //Step 1// aims to achieve. |
| |
| That means the new part is the neuro-adaptive component. Rather than try to toss the entire thing in, it is better to implement neuro-adaptive estimation on a simpler system as the only adaptive component. In fact, it is best to first try out the neuro-adaptive controller on a first-order scalar system. The one below is a great option: | Assuming that you know how to implement a neural network, then the new part is the neuro-adaptive component. |
| | Otherwise you've got two new parts, the neural network and the neuro-adaptive controller. Continuing, rather than try to implement the more complex version of a neuro-adaptive controller for a multi-state system, it is better to implement neuro-adaptive estimation on a simpler system as the only adaptive component. In fact, it is best to first try out the neuro-adaptive controller on a first-order scalar system. The one below is a great option: |
| \begin{equation} | \begin{equation} |
| \dot x = f(x) + u(x; \alpha) = f(x) + k x - \alpha^T \Phi(x) | \dot x = f(x) + u(x; \alpha) = f(x) + k x - \alpha^T \Phi(x) |
| This is just one tip. Overall, if you are not doing it, you need to learn how to break down a problem into digestable bits. The Step 1 and Step 2 breakdown does that, but you can and should go ever further yourself when resolving this project. | This is just one tip. Overall, if you are not doing it, you need to learn how to break down a problem into digestable bits. The Step 1 and Step 2 breakdown does that, but you can and should go ever further yourself when resolving this project. |
| |
| ==== Tip: Implementing the Function Approximator ==== | ==== Tip 2: Neural Network Approximation ==== |
| | |
| | Suppose that you haven't ever implemented a neural network. Well, then before even starting the neuro-adaptive part, it is mission critical to understand how to construct and approximation functions using a neural network outside of an adaptive controller. That means taking arbitrary nonlinear functions that you like and building neural networks that approximate well the functions. Play around with the neural network parameters: the number of neurons, the bandwidth of the neurons, etc. Go from functions of a real variable to vector input functions, one dimension at a time. See how the increase in dimension increases the complexity of the network (or rather the number of neurons). |
| | |
| | Overall, there is an exploration path that you need to follow if you are going to succeed at this. |
| | ==== Tip 3: Implementing the Function Approximator ==== |
| |
| One difficult thing for many newbies to Matlab is writing compact and efficient code. Efficient usually means avoiding ''for'' loops as much as possible. Chances are your neural network will have on the order of 100 to 10,000 //neurons//, so you really want to leverage Matlab's built-in functions for iterating over a matrices. You want to do the same for the center creation. Some Matlab function shout outs are: | One difficult thing for many newbies to Matlab is writing compact and efficient code. Efficient usually means avoiding ''for'' loops as much as possible. Chances are your neural network will have on the order of 100 to 10,000 //neurons//, so you really want to leverage Matlab's built-in functions for iterating over a matrices. You want to do the same for the center creation. Some Matlab function shout outs are: |
| If you do it all properly, the addition of a neuro-adaptive controller shouldn't add but about 4-5 lines of code to the setup prior to invoking the ''ode45'' part, about 3-4 lines of code to the actual differential equation function code, plus a 3-6 line function that computes $\Phi(x)$. For sure you should be writing a separate function for $\Phi(\cdot)$. In all, there shouldn't be more than a 20 line code difference between the traditional MRAC and the neuro-adaptive MRAC systems. I write this to help you try to get compact code that will run more efficiently. If you don't do this, then the numerical integration can take a //very, very// long time relative to doing so. In my version, the neuro-adaptive controller runs maybe 30% slower than the MRAC one. Simulation out to about 150 seconds takes about 15 seconds or less (I am just going by feel as I didn't keep track). | If you do it all properly, the addition of a neuro-adaptive controller shouldn't add but about 4-5 lines of code to the setup prior to invoking the ''ode45'' part, about 3-4 lines of code to the actual differential equation function code, plus a 3-6 line function that computes $\Phi(x)$. For sure you should be writing a separate function for $\Phi(\cdot)$. In all, there shouldn't be more than a 20 line code difference between the traditional MRAC and the neuro-adaptive MRAC systems. I write this to help you try to get compact code that will run more efficiently. If you don't do this, then the numerical integration can take a //very, very// long time relative to doing so. In my version, the neuro-adaptive controller runs maybe 30% slower than the MRAC one. Simulation out to about 150 seconds takes about 15 seconds or less (I am just going by feel as I didn't keep track). |
| |
| ===== Step 3: Matched and Unmatched Uncertainty ===== | |
| |
| The inverted cart pendulum system has a set of nonlinearities that are matched, plus a set that are not. Consider adaptive control for that set: | |
| |
| \begin{equation} | |
| \dot x = A x + f_u(x) + B(\theta) (u + f_m(x)) | |
| \end{equation} | |
| |
| Instead of using a GRBF-NN, consider cancelling only the part that can be cancelled and leaving alone the part that can't be cancelled. Manipulate the equations of motion to get the form of $f_u(x)$ and or $f_m(x)$. What are the basis functions in $\Phi(\cdot)$? Come up with an adaptive controller for the unmatched uncertainty by considering it to be a non-vanishing disturbance and compensating for it appropriately. Implement it and perform the proper simulations and comparisons based on the earlier steps. | |
| If you are taking the Nonlinear Control class and have covered ISS, then you should be able to come up with a targeted adaptive controller similar to the disturbance rejection one that compensates for the state-dependent unmatched uncertainty with better boundedness or stabilization properties relative to naive adaptive disturbance compensation. | |
| |
| |
| **Note:** The uncancelled parts can actually be recovered using ideas from concurrent learning. Basically store the evolution of the system and use it to recover what that part should look like, assuming that the remainder of the adaptive controller was functioning properly. As data gets collected, one can perform standard regression on the data. It is even possible to try to correct for that nuisance term, but doing so is outside of the scope of this class. Just letting you know it is possible, not part of the actual step. | |
| |
| ====== Report Considerations ====== | ====== Report Considerations ====== |
| ---------------------- | ---------------------- |
| |
| Even though the neuro-adaptive controller was done first, a sequencing that makes sense goes from no adaptation to matched adaptation and eventually to neur-adaptation. I might be mistaken, but the neuro-adaptive part should have some ability to correct for the unmatched since those dynamics are influencing the error. To support each of these parts, the report should include the appropriate controlled equations of motion for the different realizations (linear, nonlinear, matched+unmatched, etc). It should cover the controller design and control synthesis for static and adaptive cases. If attempting an adaptive structures slightly different from what was covered in the lectures, then its derivation should be included. If using adaptive controllers covered in class, then only their setup and final adaptive laws should be covered. Trajectories applied should include regulation (moving to a new, feasible set point) and tracking. Just like in homeworks, attention should be paid to highlighting how the static controller fails to perform under incorrect parameters estimates. Otherwise, the //Final Deliverable// assignment item should cover what's needed. | Even though the neuro-adaptive controller was done first, a sequencing that makes sense goes from no adaptation to matched adaptation and eventually to neuro-adaptation. I might be mistaken, but the neuro-adaptive part should have some ability to correct for the unmatched since those dynamics are influencing the error. To support each of these parts, the report should include the appropriate controlled equations of motion for the different realizations (linear, nonlinear, matched+unmatched, etc). It should cover the controller design and control synthesis for static and adaptive cases. If attempting an adaptive structures slightly different from what was covered in the lectures, then its derivation should be included. If using adaptive controllers covered in class, then only their setup and final adaptive laws should be covered. Trajectories applied should include regulation (moving to a new, feasible set point) and tracking. Repeat runs are a must to see how the adaption influences future performance (use the adaptive parameters from the previous run as the initial conditions for next run). Just like in homeworks, attention should be paid to highlighting how the static controller fails to perform under incorrect parameters estimates. Otherwise, the //Final Deliverable// assignment item should cover what's needed. |
| |
| ====== References ====== | ====== References ====== |
| ---------------------- | ---------------------- |
| |
| No references given. This project is self-contained given the lecture notes and the tips. | No references given. This project is self-contained given the lecture notes and the tips. The Hovakimyan and the Lavretsky and Wise references also work. |
| |
| ------ | ------ |
