Wednesday, March 23, 2011

Real-time Controlled Robotic Arm

To finish up my B.S. in Computer Engineering, I had to come up with a design project. I had always wanted to create a robotic arm and control it with a joystick or something. I had also recently been playing around with the open source 3D program, Blender 2.49 (this does not work in Blender 2.5+).  After realizing the potential of Blender's built in Python scripting capabilities, I figured why not let Blender control it! Blender can use the full Python installation, Python can send data out the USB port, and there is dozens of ways to control a motor from a microprocessor. I figured I could use Blender to create a 3D model of the robotic arm, control the arm inside of Blender, and have the physical arm stay in sync. So I did!




Initial Testing

My initial test was to control a single servo by rotating an object. This was the result (sorry for the bad quality). It can control it with either a key-framed animation (as seen in the first part of the video) or in real time by moving the mouse (as seen in the latter half).



This was accomplished using Blender, Python, and an Arduino. Blender 2.49 has a Python interpreter built into the program itself. It is able to read (and write) values from the 3D space (along with numerous other features). Blender can also be set to call a script any time the 3D space is updated (ie something moved, rotated, ect). So I attached a script to Blenders update call back. Any time the object was moved, the Python code would get the object, read its Z(?) rotation, and send it out the USB port. By default Python cannot write to the port, but the pySerial module for Python allows it to write to a serial port (I believe for Windows, pyWin32 is also needed). The Arduino driver code makes the USB port look like a serial port. Thus, Python could write (and read if needed) values to the Arduino over the serial port. (There are numerous examples online of having Python and the Arduino communicate). Python would write the angles of the object anytime it was updated. As soon as the Arduino got these values, it would rotate the server to this angle. The Arduino used the Servo library to control the servo.

I did not save the code to the above example, but I through together some code that is probably pretty close. Here is the Python code that would run in Blender 2.49. Here is the code for the arduino.


Moving on to Bigger and Better Things

Before I was sure I would be able to control multiple servos, I set up a few tests. Much to my surprise expectations, it worked perfectly. I knew that since I could control multiple servos simultaneously, that the major part of the project would be constructing the arm. So for prototyping purposes I rigged up a 2 joint arm from cardboard and wire (pretty innovative if I say so myself). Once again this could be controlled in real-time or by a key-framed animation. (Its not perfect, but hey, its cardboard!)




The Software

From the code posted above, its not hard at all to make this work with multiple servos. Its a matter of getting multiple object's rotation, sending all of them, and having the arduino know which angle is for which servo.This could easily be done by always sending angle A, B, and then C, then the arduino would know A goes with servo X, B goes with servo Y, and so on. However, with mine, I made it a little differently. I would send a number and then an angle. That way the arduino gets two numbers, the first to indicate which servo, and the second is the angle.

I added alot of complexity to the code because I created a GUI (GUIs in Blender are not easy). It allows a port name to be typed in and connected / unconnected to when desired. Since the above code opens and closes the port EVERY time it sends a value, the connect button opens and closes the serial port when it is clicked, not every time a value is sent.

I also added the feature to enable and disable joints. This helped when debugging individual servos. It also has an offset for each joint. The 3D arm is in a default position, when the actual arm starts up, it goes to this same position. That allows the offsets to provide a means of calibration.

Here is the blender file used. It contains the Python code and the 3D arm rigged with inverse kinematics. Here is the arduino code that works with it.


3D

Not a whole lot to explain. It would be best to download the blender file just above and play around with it. Its just simple modeling and rigging (parenting, bones, etc.). Below is some demonstration of the rigging.


 

Hardware

For the construction of the arm I used aluminum. (Not only is aluminum light, it is also pretty easy to work with). Hobby servos are usually pretty cheap but the stronger you need, the more expensive. Since this arm has 3 pivots, a gripper, and a rotating base, the lower joints had to be extra strong. For the lowest joint I used 2 HiTec HS-645MG servos (kind of expensive). They worked together to provide sufficient torque. For the next joint up I used one HS-645MG. For the grippers joint, I used a HiTec HS-422. For the gripper I used the HS-422 to power a gripper. For the base (to let it rotate left and rigt), I used use another HS-645MG. Here it is half way constructed.


Another major part was the electronics. I used a roboduino (an offshoot of the Arduino made for servo control). In total there were 6 servos, and they all could be moving at the same time, some of them under a decent load. The issue was that the arduino is not capable of handling that much current, even powered off of an external supply. The boards circuits are two small. (Some of the larger servos used can draw almost 1 amp). So I built an AC to DC converter and used regulators for the servos. I also powered my arduino from the regulated voltage. The servo's power would come directly from the converter, not through the arduino. The black box in the pictures housed all the electronics. Pictures below.

(I'll eventually add a video with it fully complete...someday...)

Pictures!!!

14 comments:

  1. Justin, That's fricken amazing! Congratulations on your success. You've just tipped me over the edge of learning Blender. Been wanting to learn it, but wanted to wait for 2.5 (will your blender file work with 2.5+?)

    Seriously cool stuff!

    ReplyDelete
  2. Nice work there Justin, I love the fusion of real world with the virtual

    Sam

    ReplyDelete
  3. @Dan The .blend file will open in 2.5. The rigging seems to work fine. However, the Python scripting WON'T. Blender 2.5 has a totally new API. :(

    @samiam Thanks! That's the goal I was aiming for!

    ReplyDelete
  4. Oh bugger, still I assume it's still possible to do with blender 2.5?

    BTW - I finally got around to building a bezier curve editor in processing. http://vimeo.com/19052860 still a WIP but it's getting there. The core functionality is ripped straight from the blender source code :)

    ReplyDelete
  5. I'm sure if the script was rewritten using the Blender 2.5 API it would work.

    That bezier curve is amazing! At one time I looked into that part of Blender's code because I was needing to solve for a specific X axis value of a bezier curve. (As I recall it has to solve a quartic function I believe). However this turned out to be too intensive because I was needing to do it on an Arduino for Kevin Larson.

    ReplyDelete
  6. Yeah I remember him mentioning that. Kinda got lucky with the bezier curve. A guy at work who was clever enough to port the blender source to lua and then gave it to me to port to processing! :)

    Still I have no idea how the function works, but it's great because it's not and approximation like all the examples out there and there are no iterations on the curve to get the result! Can't wait to flesh out a simple gui that might be useful to some people.

    ReplyDelete
  7. This is really a nice project! Congratulations!

    ReplyDelete
  8. I have been trying to recreate this project, but I have been having a few problems. How do the arduino and blender work together?
    How can you use the link that you gave for the individual servo and python if there is no where in blender to add code?
    Also, would you recommend aluminum over plexiglass?

    ReplyDelete
  9. I just managed to figure out where to plug the code into blender. I was reading your page again, and I realized that arduino allows Python to run on a serial port because it can't run on a USB. That is why pySerial makes it run on a serial. I have a problem though because I am using a Macbook, and it does not have a serial port. Since Python cannot run on USB and I do not have a serial port, how am I supposed to run the code, which is searching for a serial port? Even if I try to use a serial/USB convertor, it does not work because I need to use a USB port on my computer (which does not work for Python).

    ReplyDelete
  10. The Arduino drivers make the USB port look like a serial port. And the pySerial also sees USB ports as serial ports. So your actual computer does not need serial ports on it.

    ReplyDelete
  11. Well done and thanks for sharing your work.

    ReplyDelete
  12. Great job, Justin! :) Not meaning to upstage you, I actually thought of that 2 years ago, but didn't have the time to get up to speed with Python skills back then. So we fell back to MATLAB which we were familiar with. Well, Blender still had the part to play, that of creating the 3D model. Wonderful for you! Here's a link to a YouTube video of our project http://goo.gl/Zl10t. You did a whole lot better with the real time stuff :)

    ReplyDelete
  13. Hey Justin,

    Finally go around to releasing a first iteration of that bezier curve editor I mentioned in the comments above. Here's a link: http://danthompsonsblog.blogspot.com/2011/08/my-first-animation-editor-with-bezier.html

    ReplyDelete
  14. If you where a woman.........I would marry you. Then I'd have it all.

    http://www.blendernation.com/2009/04/10/blender-for-robotics-is-shaping-up/

    ReplyDelete