Eric Dubberstein
Weekly updates for 3/23:
Gitbook has refused to save my work multiple times because there are too many open change requests. For now, I will be using google docs for my weekly update until A-V approves my change requests.
Read my gitbook update below (hopefully this saves)
https://docs.google.com/document/d/1t3JHU0V2qEjshk2x43ekmjDYdkZ4ChMUf5-mbo72TLA/edit?usp=sharing
I tried to paste the google doc below:
Gitbook has discarded my work multiple times because there are too many change requests.
My update is going to be on google docs for now.
Progress for this week:
I got local job creation working on the arduino.
Here is a video demo:
Note that the job doesn’t actually do anything. Anirud is going to have the spincoater set up tomorrow, so I will be able to test it for the next week’s demo.
https://drive.google.com/file/d/1zMtn7b1HO5kucqc2ypLdfpWQ1Wd0AuPt/view?usp=drive_link
All of the changes to the lab_com_gui.py file are in the repo (under the EricLabComV1 branch)
Here are the highlights of the changes:
New GUI buttons added. Note that these buttons are NOT hardcoded.
Instead, the buttons are configured in one spot near other parameters that will be changed when a new tool is integrated. I am still improving this system. The documentation will be improved for this once it is finalized.
Lastly, there is some logic to show and hide the UI elements as appropriate and complete the request after the user types in the details.
Note that if the RPI is not connected to the database, the job will still be run locally.
The documentation is up to date (although it will be improved once I have finalized the code structure).
The gitbook project tracker is up to date.
I am unable to complete the tasks from week 8 outlined below (testing the full system with the spincoater) because arinud did not finish making the mock spin coater yet. He said he will have it completed by tomorrow (3/24) and I will be able to test the full system then.
Plan for next week:
original tasks from week 8 since I will have the spincoater to test on:
Week 8: End-to-End Testing
Test the full data flow:
Create a job using Postman.
Fetch the job on the Raspberry Pi.
Execute the job on the spin coater.
Verify job completion status is updated in AWS.
Debug and address issues in data flow, timing, or physical connections.
Justification: This will be the first test of the whole system.
Tasks from Week 10:
Week 10: Optimization
Optimize the Raspberry Pi code for performance and reliability:
Handle API timeouts or failures gracefully.
Add retry logic for API calls.
Refine the UI for user-friendliness based on feedback from initial users.
Justification: This will be important for future scalability.
In addition, I will meet with stepper team on monday 3/24 to look into integrating this system with the spincoater.
Feedback from documentation/presentation:
The feedback for the documentation and presentation were positive, but there are a few things to work on moving forward:
Improve communication with other teams, specifically regarding need for "real-time" data gathering. AV: Who should I meet with in the lab to discuss this?
Improving documentation specifically related to how to implement a new machine into the framework: I will make sure to do this as part of the tasks in week 11.
Roadblocks:
Need access to spincoater (Anirud will have it done by EOD tomorrow, 3/24)
Need to get in contact with other teams who want a continuous data stream (this is question for AV: who should I reach out to?)
Need to have gitbook changes accepted & cleaned up.
Weekly updates for 3/16:
Progress for this week: Existing arduino code for spincoater firmware is integrated with the USB communication to the RPI.
Here is the updated driver code that runs on the RPI:
This code is run on the RPI to control the spincoater. It simply sends messages over UART with the RPM value, the time value, and the start command.
Here is the code that will run on the arduino. Notice that only a section was added for the USB UART interface.
ROADBLOCKS:
I need to get access to an additional spincoater to test my design. Arinud said he can rig one up quick. I will reach out to him this week to get this completed.
Plans for next week:
The project is going pretty well (on schedule), so I am going to tackle two weeks this next week.
Week 8: End-to-End Testing
Test the full data flow:
Create a job using Postman.
Fetch the job on the Raspberry Pi.
Execute the job on the spin coater.
Verify job completion status is updated in AWS.
Debug and address issues in data flow, timing, or physical connections.
Justification: This will be the first test of the whole system.
Week 9: Redundancy and Local Job Creation
Add functionality to the Raspberry Pi UI to:
Allow users to create jobs locally.
Optionally enqueue these jobs in AWS.
Test the local job creation and execution workflow.
Justification: Local job creation is a nice feature to have but not absolutely necessary, so it is put towards the end of the schedule.
The goals in week 8 will be simple to accomplish because it just requires testing once I get access to the additional spincoater. I want to have these accomplished for the second demo.
Thus, I will work on local job creation on the RPI in parallel.
Documentation and project tracker is also updated.
Weekly updates for 3/2:
Progress for the week: Arduino connected to full system. In other words, we can now control the arduino via AWS requests.
Important details:
I spent 5+ hours attempting to get the raspberry PI to arduino connection to work using the GPIO uart TX and RX on the rasberry pi. Ultimately, I was unsuccessful and realized that it would be much easier (and scalable in the future because we have multiple USB ports) to simply connect the arduino via USB and use the UART through the USB cable.
Here are the reasons why we are using UART over USB instead of using the GPIO TX and RX on the rasperry PI and arduino. (Also included in documentation).
The raspberry PI runs on 3.3V while the arduino runs on 5V. This presents a problem for the signal from the arduino to the RPI. We have to use either a level shifter or a voltage divider. I didn't have a level shifter on hand, so I tried building a voltage divider using a 20k and 10k resistor. This divided the voltage as needed, but it's possible this was part of the issue. Here is an image of the basic voltage divider I tried.
I also attempted to bitbang a UART connection between the RPI and arduino. This was also unsuccessful. After a bit of research, it looks like this is because Raspberry PI OS is not a RTOS (real time operating system), so the timing is not precise enough to bitbang a UART connection.
At this point I was pretty stumped, so I searched around the internet a bit. I then realized it would be MUCH MUCH easier to simply use UART over the USB connection between the two devices. The reasons why I settled with this approach are as follows:
I was able to get UART over USB from RPI to arduino working, but I was not able to get it working using the GPIO UART module.
This solution is more scalable as we can have multiple arduinos that control individual devices plugged into one raspberry PI as the RPI only has one UART TX/RX vs multiple usb ports.
We can leverage existing consumer grade USB extension cables and hubs
The USB cable is physically studier than the multiple thin GPIO cables. Also, there is only one cable to deal with.
Here is the code that runs on the arduino:
NOTE: This code will eventually be integrated into the existing spincoater firmware to control the spincoater instead of just turn the LED on and off.
It does the following things:
On startup, it sets the baud rate (speed of UART connection) to 115200 and sets up the LED pin output
It reads an entire command from the RPI
It parses the integer number of seconds to run the LED from the command
It turns on the LED for that amount of time
Here is the updated lab_com_gui (running on RPI):
I've only included the part of the code that changed.
The first portion that changed just opens the UART port to communicate with the arduino.
The second portion sends the command to the arduino. Note that it is a self-contained method for this new device. To add this device to the code, all we needed to do was add the peripheral config and write this method.
This method also reads out the debugging messages from the arduino. Once we actually get the spincoater working, this could be messages if the spincoater is malfunctioning.
Video demo:
The documentation is updated.
Github progress tracker updated.
Roadblocks:
I need to reach out to anirud to make sure that it is ok that we are interfacing with the spin coater arduino via USB. I am assuming this will be fine, but I still need to check with him.
Permission to test on the spincoater or a duplicate spincoater. I will talk to anirud about this once I get the code working on the test arduino.
We need more USB cables for the arduinos. I had to search through hackerfab until I found one, but I am afraid someone else might need it eventually.
Plans for next week (spring break + week after):
Week 7: Microcontroller Integration
Complete mid semester documentation.
Finalize the physical connection between the Raspberry Pi and spin coater.
Write and test code on the Raspberry Pi to interface with the microcontroller:
Ensure commands (e.g., start, stop) are executed reliably.
Justification: This is crucial for automating the spin coater.
NOTES: This will not involve the actual spincoater, but rather the arduino programmed with the spincoater's firmware. Anirud is building a new spincoater which we will test on later in the semester.
Hmm, the shift to USB should be fine, and lmk I can order some more USB cables if needed. I think USB switch is good, since we need to have a standard and using GPIO might have been less accessible than UART. But the arduino control through postman requests is big, looking forward to seeing the spin coater work with the full end-to-end system! Also as we spoke, I will see how we can best put this up on the website, whether it's a new Django page or interfacing w Ridge's prototype.
Weekly updates for 2/23:
Nice work. I am def interested in using the GUI, although I don't have a linux machine on me so I might have to borrow the RPi from u haha. As we discussed on discord, feel free to create a folder for the files you write, and then just put that link in this update and the documentation, so that everything is more concise and you don't have to paste your script in multiple places, like here, db documentation, etc.
By the way great work keeping the documentation updated. I would say add some more explanation of how the code works, maybe some pseudocode or some high level explanation that tells people what is going on with each new step.
Progress for this week: I wrote a Tkinter GUI for the raspberry PI. Here is a demo video:
https://drive.google.com/file/d/1urpoj3qYbh18Ti-hXBs-8LBYLlDNa4fD/view?usp=drive_link
The python code is below. Here is the key piece: only one function will need to be modified to make it work with a different tool (it is clearly marked in the code with comments). This is critical for extensibility of this system. The code was written with about 20% chat GPT (it took care of all of the boilerplate code), but it needed significant tweaks by hand.
This took longer than expected with testing, so the local lob creation has been pushed to week 9 (this task is listed in both weeks 5 and 9 on my roadmap).
The database section of the documentation has also been updated.
Tasks are updated in project tracker.
Roadblocks:
I need to meet with the spincoater team to get started on interfacing with the spincoater. Solution: I have a meeting scheduled for Tuesday afternoon.
Plans for next week:
Week 6 tasks (Some of these may bleed into week 7, but this is planned for in the project plan roadmap)
Write Python code to send job parameters to the spin coater microcontroller:
Simulate button presses using GPIO pins (might be better way to do this, will talk to spin coater team)
Implement monitoring for job execution and update the status locally.
i.e. store the status of the current job locally before sending to AWS.
Integrate the POST /job_completion API call to report job results to AWS (this is already complete at the start of the week!!)
Justification: this is the setup week for week 7. I will also try to start those tasks if I have time as those could be time consuming.
Weekly updates for 2/16:
The first thing I did this week was get a basic API test script written. See below. This script will be useful to ensure that the api is working if we make modifications in the future. It follows the standard job lifecycle.
As you can see, it is currently passing.
The next thing I did was all the work I had originally planned for next week (i.e. have the python script pull data from the AWS database when running on the raspberry pi)
See this video for a demo. I am not talking during it because this is the video I plan to use for my presentation. I'll bring the raspberry pi to class for the presentation also, but I think it will be easier to just play the video and then I can focus on explaining it and everyone can see what I am doing much more clearly.
thus, I've accomplished all of my goals that I had for myself this week
Roadblocks:
No major roadblocks at this time. I will need to get in contact with the person in charge of the spin coater microcontroller code next week to start planning out how I will do the integration with the real spin coater. I don't recall his name, but I will get the ball rolling on this after class on tuesday.
Tasks for next week:
For next week, I will start tackling the jobs in weeks 5 & 6 (I'm a week ahead!!)
Week 5 tasks (main focus. Will get these 100% done):
Develop the GUI using Tkinter:
Display job details fetched from AWS.
Add buttons to start, stop, or skip jobs.
Include an interface for creating new jobs locally.
If Tkinter GUI does not work well, write command line interface instead.
Test GUI on the Raspberry Pi screen.
Justification: GUI will be important for further testing. GUI development can take time so I set aside a whole week.
Week 6 tasks (will start working on these, but might not complete by end of this week. Key idea is to start working with spincoater team)
Write Python code to send job parameters to the spin coater microcontroller:
Simulate button presses using GPIO pins (might be better way to do this, will talk to spin coater team)
Implement monitoring for job execution and update the status locally.
i.e. store the status of the current job locally before sending to AWS.
Integrate the POST /job_completion API call to report job results to AWS (this is already complete at the start of the week!!)
Justification: this is the setup week for week 7. I will also try to start those tasks if I have time as those could be time consuming.
Weekly updates for 2/9:
Tasks I accomplished:
Database is set up and configured for api endpoints that are needed for the first demo. These are POST /jobs, GET /jobs/next, and POST /job_completion. Full details of the implementation can be found in the database section of the documentation
Demo: Just photos for this week of sucessful api calls.
Roadblocks:
No significant roadblocks. I just need to pick up the cable that got delivered before using the monitor
Plans for next week:
From schedule:
Use Postman to test all API endpoints for job management.
Populate DynamoDB with test jobs and verify the job lifecycle (enqueue → in progress → completion).
Create a few basic automated tests
Justification: Automated testing is extremely important for future reliability.
Additional tasks:
Although it is officially scheduled for week 4 on the project proposal, I will make sure I get a basic API fetch working on the raspberry pi to support my demo. My demo will have me sending an API request to postman on my computer that then turns on an LED (or changes the voltage on a multimeter).
Good work, I am wondering how this would work beyond the API get/post and postman interface? As in how does it connect to a low-level machine, I am assuming since it's a HTTP request it can be handled by something running on a port on the local machine somehow, but I am curious to find out more about how this is done.
Weekly updates for 2/2:
Tasks I accomplished:
Physical hardware is set up. Raspberry PI is connected to spin coater microcontroller. I am able to send data to microcontroller using the Raspberry PI GPIO pins. I checked off these tasks in github progress tracker.
See database section of gitbook for documentation.
Weekly demo:
https://drive.google.com/file/d/1g-vEBh8HtQuMvDOqQpFuI3LrrTV60lJq/view?usp=drive_link
Roadblocks:
I do not have a micro hdmi to mini hdmi cable needed to plug the portable monitor into the raspberry pi. Solution: This has been ordered on AMAZON and will arrive within the next week or so. For the meantime, I can just connec the cable to the monitor at my desk.
Plans for next week:
Set up AWS resources: Create DynamoDB tables for the job queue and logs.
Configure S3 bucket (if needed) for job-related resources. Implement basic API Gateway endpoints using Lambda:
POST /jobs: Enqueue new jobs.
GET /jobs/next: Fetch the next job.
POST /jobs/completio
n: Update job completion status.
Deliverable: Can enqueue a job from POSTMAN on any computer to turn on the led (or change voltage on multimeter) connected to the GPIO pin on the raspberry PI.
Good work. I think the cable should arrive by tomorrow or so, I will let you know. If you are stuck on AWS stuff let me know. I would say just create a free account, if not I can also give you access to the HackerFab account if needed. I like the weekly demo thing, I am going to suggest that to others as well. Now that the GPIO pins can be used to send data to the microcontroller, maybe you can even try modifying the microcontrollers spin-coater script actions based off the data received to the microcontroller from the raspberry pi. If this is the next step u are working on, that's great.
Mid week update 1/30:
I got a basic test script running on the raspberry pi to turn a GPIO pin on and off
Roadblocks: There is no mini hdmi to mini hdmi cable to plug the raspberry pi directly into the portable monitor
I need headers to be able to connect the wires to the raspberry pi and the arduino (for the spincoater).
Weekly updates for 1/26:
My primary deliverable for this week was creating the plan for the rest of the semester for my role on the database team.
Here is the proposal document: https://docs.google.com/document/d/1SrD66bqmS1xxs2jt_WWSY7eLr6vyDJxIB4dbMdbxkws/edit?usp=drive_link
My roadblocks are as follows:
The parts from the BOM outlined in the proposal document need to be acquired. Most of the parts should be ready to use by the end of class on Tuesday
I do not think there is a spot on the github documentation for the database team. I will ask my teammates on tuesday to get clarification on where to document how the system I am building works. Right now, all that documentation is done in the proposal document.
Next week, I will follow week 1 of the project proposal document. Specifically, this is:
Week 1: Hardware Setup
Assemble and configure the Raspberry Pi 5:
specifically:
Install Raspberry Pi OS on the microSD card...
Connect and test peripherals (monitor, keyboard, mouse).
Ensure the Raspberry Pi can connect to CMU WiFi or Ethernet.
Prototype physical connections between the Raspberry Pi and the spin coater's microcontroller using the breadboard and jumper wires.
NOTE: there is an extra spin coater microcontroller that I will be able to interface with
Weekly updates for 1/19:
On thursday after class, I met with the rest of the database team. We got the repo cloned on all of our machines and we outlined what we need to accomplish by the end of the week (1/19). Our task was to browse through the current working version of the website running locally on our machines and make a list (here in gitbook) about what could be improved as a new user that has not used the website before. My list is as follows:
IDEAS FOR IMPROVMENTS:
On the homepage:
change the name of the button "chip summary" to "See everyone's chips" or "All user's chips", etc. As a new user to the website I wasn't sure what "chip summary" was before I clicked on it.
When I click "Input"
Then chose AluminumEtch from the dropdown menu and click submit,
I get the error as shown in the screenshot above. This is likely an edge case because I haven't added any chips yet.
When I instead choose create a new chip, I am directed to this page:
This page also is unintuitive as a new user. Why are there two submit buttons? My understanding is that you type in the number, hit submit, fill in the rest of the details, and hit submit again? All for the same chip? This is unintuitive and the first and second parts should be separate. Finally, when you hit the final submit button, there is no clear feedback that the data has been submited. The text is just cleared. There needs to be some feedback to the user (even just plain text that says "data submitted") when the form is submitted.
There should also be a clear back button to go back to the main menu.
Next, I went back to the homepage and clicked on search.
Here we have the same issue where we have two submit buttons after submitting the first form. It's not clear to the user what is supposed to be done. The first menu should be hidden after the first submit button is clicked.
Next, I typed in a search chip number and clicked submit. I am then brought to an error page. This is clearly an issue that needs to be addressed.
It is also generally unclear what we are searching for when using this search tool.
My plans for next week are TBD and will be decided after class on tuesday. It will likely include fixing a subset of the bugs I identified above.
Last updated