I had this idea of creating a collaborative AR experience where friends could draw and create art together. I made a small prototype just to play with the format and how it would feel on an iOS device while learning to combine RealityKit and the pretty amazing PencilKit. At this point it mostly serves as a demo of how to integrate RealityKit and PencilKit in a nice way but who knows what will happen in the future. Read more

I once worked on adding support for hands tracking for an existing VR game on the Oculus Quest2. Moving from controllers to just using your hands made me think about interaction design in VR/AR in a new way. The dream scenario would be to just do the same thing as with controllers, but with hands. That would be fast, cheap and fancy, right? Turns out it worked but wasn’t very fun, and didn’t spark joy. Read more

I love animation systems. I wrote animation systems for the C64 and Amiga in the 80’s and 90’s, for fashion designer apps at H&M, for games like Candy Crush and god knows how many more. RealityKit has a nice and user friendly animation system that can handle different linear world space transforms, play model space animations from usdz models and combine them in sequences or groups. RealityKit entities can play an AnimationResource with the playAnimation method and AnimationResources are created from AnimationDefinitions. Read more

We spent last post learning how ARKit anchors work and how they relate to RealityKit entities. In the end we wrote our own RealityKit AnchorEntity based on the results of the ARSessions provided ARAnchors. This is all fine but if we just need a quick anchor we can use the RealityKit provided AnchorEntity. We start the same way by setting up an ARSession. Requesting horizontal plane detection. private func setupARSession() { let session = self. Read more

I’ve spent a lot of time learning and writing code based on RealityKit. I really like the framework. It’s nicely written, has a lovely tiny, lightweight ECS implementation and makes writing AR applications and games pretty straightforward. I thought I’d write down some things I’ve learned in a series of blog posts that might help other AR devs out there in the wild. The first thing that comes up when writing a new RealityKit app is detecting surfaces in order to anchor virtual objects in the real world. Read more