3.24.2016

All About the sARndbox: An Augmented Reality Playground


Update: sARndbox "In Plain English" video

Our kiddos made their own video explaining the AR sandbox :) 



I've been posting a lot on social media about our latest makerspace creation and so many people (okay, 2) have asked how we did it (and provided a lot of encouragement on the days I thought it might fail - thanks #TCEAtribe). Hopefully this blog post answers most of the questions. Once I go through all of our video footage (the kids took over 4 hours of footage), I'll post some more videos on our process. As always, if you have any questions or need clarification, please just reach out to me at @LibrarianLister

I wish I could say we came up with the idea, but we definitely didn't. Oliver Kreylos, a computer scientist studying 3D scientific visualizations and computational geosciences at UC Davis designed and programmed the AR sandbox software. He made the software available as open source - YAY! - so we, and YOU can go grab it. #GoOpen. We first learned of these sandboxes at the Young Innovators Fair in Philadelphia this January. Myself and another teacher took a group of 20 very curious middle schoolers to this event over our Winter break. It was phenomenal.

Shortly after the Fair, we got asked to present something relating to makerspaces at the Taste of Hershey. The Taste of Hershey is our school's foundation's biggest fundraiser. A bunch of restaurants and businesses come together for a day of fun conversation and great food. 

There are a lot of resources out there on Augmented Reality sandboxes. Of which, I understood none completely. If it weren't for my genius, computer-programming and mechanically inclined students, I would truly have a box with sand. 

This blog post is an attempt to put the many resources into plain English and I'll provide links along the way so you can see more. 

WHAT IS IT?

There are quite a few videos out there about the AR sandbox (trust me, we watched them all!) but this one does a great job of just answering the big "WHAT IS IT" question.


HOW DOES IT WORK?

AR Sandbox uses a computer projector ​and a motion­ sensing input device​ (an Xbox Kinect) mounted above a box of sand. The Kinect detects the distance to the sand below, and a visualization ­­ an elevation model with contour lines (aka. a topographic map) and a color map assigned by elevation ­­ is cast from an overhead projector onto the surface of the sand. As the sand is moved, the Kinect perceives changes in the distance to the sand surface, and the projected colors and contour lines change accordingly. When an object (for example, a hand) is sensed at a particular height above the surface of the sand, virtual rain​ appears as a blue, shimmering visualization on the surface below. The water appears to flow down the slopes to lower surfaces. 


HOW IT'S MADE?

Here are the must-have's to create your own AR sandbox:
  • A sandbox that is a 4:3 ratio (this is so it matches the ratio of the projector). It's not a big deal if your box is bigger than the 4:3 ratio, just know that your projection won't fill the entire box. You can either build your sandbox out of wood (what we did) or find something to use as a makeshift sandbox (think dresser drawer or a plastic storage bin).
    • Ours was 32" x 24". In hindsight, I might have made it a little bigger but since we did a lot of transporting, I'm glad ours was smaller.
    • If building your sandbox out of wood (as we did), I'd recommend water sealing it or lining it with a plastic tarp of some sort. Ours is not a permanent fixture so we did neither.
    • TIPS: Your INTERIOR measurements are what's important. So if you're using 2" thick wood for your sandbox walls, you'd want to factor that in when building the box.
  • Caulk to seal the inside of the sandbox. I guess this wasn't super necessary but our kids liked using the caulk gun.
  • Sand that will cover your entire sandbox with at least 4" of sand.
    • We used Play Sand from Home Depot and added a little bit of water when we did the demonstrations. This way we could do a bit more molding. We used 3, 50lb bags ($3.90 x 3 = $11.70)
    • Some places recommend white sand such as this kind from Sandtastic but its much more expensive ($31.99 x 6 = $191.94)
    • You can also use Moon Sand ($58.04 x 8 = $464.32) or Kinetic Sand ($49.99 x 14 = $699.86) if you want a crazy experience.
  • A computer with at least the following:
    • Running the 64bit version of Linux
    • At least an Intel i5 (preferrably an i7 processor) running at least 3GHz. The better the processor, the faster the Kinect can scan and simultaneous project what it's rendering. Ours was about a 3 second delay because of our 5 year old i5. We found an i5 processor in an old computer in the technology "death room".
    • 2 GB of RAM to run the software. 
    • A hard drive with at least 20GB free for the installation of Linux and the software
    • A sweet graphics card. We used a GTX 950 ($177.99) but I would not go any lower than that. Ideally, go with the 980TI ($619.99) as it will look much more like the quality in the video above. We have a lot of students that build their own computers for fun. One of our students that was instrumental in building this had a 950 that we used.
  • A first generation xBox Kinect for the Xbox 360 (not the Xbox One!) A lot of kids have upgraded to Xbox One so we had several kids bring in their "extra" kinects.
  • A short throw projector. (Teachers: Think ceiling mounted projector for SMART board vs. the ones you'd set on a cart in the back of the classroom). Originally we used an old projector from the technology "death room" but the bulb died the day before the "show" (#storyofmylife). So much to the displeasure of my technology department, I stole the one from my library classroom. (#sorrynotsorry)
    • You'll want a projector that has HDMI or DVI that way you can take full advantage of the graphics card. You can still use a projector that has a VGA component, but it will definitely cut back on the quality output.
  • Something the mount the projector and Kinect above the sandbox.
    • Your Kinect should be mounted at the same height as your box is wide. (Our box was 32" wide, so we mounted the Kinect 32" high)
  • A vacuum (see images below)
  • Mountain Dew, Italian Four Cheese CheezIts ("must haves" according to middle schoolers)
  • Camera to document everything
  • Patience.
  • Perseverance.

Basically, it will look like this.


Or, if you can't read middle school, this:








MORE FROM THE ADVENTURE


Before we built the box, we built the computer. We tested the projection and 3D rendering by this very sophisticated prototype made out of masking tape (see above)

We stayed after school one evening to build the box. My husband cut the wood and did some pre-drilling. The rest was up to the kids. A few kids had never used a drill before. Boy were they ever stoked!! Once we had the box assembled, we needed to figure out the mount for the projector. 


I was super nervous about this part because it needs to be perfect. Reading through some of the help forums, I knew we needed a way to make this super study, but also somewhat adjustable. Sorry but that was way above me. We left that evening with an "eh...let's sleep on it" kind of feeling.

And then I got to school the next day and one of my students was waiting with this masterpiece:


Uh...yeah. I know, right? He took a fence post, attached one part of the projector mount to it. Then, he took a steel pipe and put it in adjustable camps to which he soldered another steel pipe so we could mount the Kinect. 

I'm not sure what made me happier - the solution to our problem, or the look of accomplishment on Joey's face ALL day. (Definitely the latter). 



The pictures above were taken before school actually started. The kids came by, saw Joey's creation, and didn't even go to their locker before they started putting it together.

Once we had everything set up, we put the box on an old laptop cart so that it was study and could house the 

The last step was calibration and boy was that a task! We spent a solid 3 hours on a Saturday afternoon just doing the calibration. And we still didn't perfect the water simulation piece. At 4:30 on Saturday, we went home praying that the thing would work tomorrow.

So on Sunday morning (raining, of course), I brought my Ford Ranger to school and packed up everything. I went to take the sandbox to my car and it wouldn't fit through the door. SERIOUSLY?! 

So we quickly moved it to a smaller cart (an old projector cart) and just prayed that it wouldn't fall. 

Eventually, we got it there.

AND. IT. WAS. AWESOME.





The community response on the AR sandbox was overwhelmingly positive. People thought this was a project for gifted and they were so surprised to learn that this wasn't a project at all, it was just a bunch of kids interested in learning more about this.

As I was editing this post, we received a phone call asking us to present at the Business Leaders Breakfast next week. I guess I'll be dismantling that projector in my library again :)


OTHER RESOURCES

  • This resource has some CAD blueprints of the AR Sandbox (in .dwg files)
  • As with every complex creation comes its own help forum. This was a life saver in our final moments.
  • This Facilitation Guide was meant for museums, but has a lot of great tips for teachers too.

I hope this helped some of you get an idea of what in the world I was posting about the past month. 


This is our next project:
A game table made from a cheap IKEA end table and a Raspberry Pi.