Video 360 in OpenGL (iOS) part 1 – 3D Scene

Video 360 blogpost header

This is the first part of tutorial showing how to create a simple player for video 360° in iOS. The player will be done in GLKit framework. There will be an OpenGL scene with a sky sphere and a first person camera. The sky sphere will have a texture mapped on it from the inside. In this part we will show how to prepare a 3d scene with a sky box (sky sphere is more complex and will be described later).

We’ll start by creating a Single View Application project. Then we’ll replace the main view controller with our custom GLKViewController called Scene3DViewController. We will also replace a default GLKView by our custom Scene3DView.

Screenshot from Xcode Interface Builder

Screenshot from Xcode Interface Builder

The main purpose of the Scene3DView is to collect scene objects (e.g. meshes) and display them.


override func display()
{
    super.display()
    glClearColor(0.0, 0.0, 0.0, 1.0)
    glClear(GLbitfield(GL_COLOR_BUFFER_BIT))

    let objects = self.sceneObjects
    for object in objects
    {
        if let renderable = object as? Renderable
        {
            renderable.render(self.camera)
        }
    }
}

Scene3DView also stores a camera object so that the camera’s aspect ratio can be updated anytime the view’s frame changes.


override func layoutSubviews()
{
    super.layoutSubviews()
    self.camera.aspect = fabsf(Float(self.bounds.size.width / self.bounds.size.height))
}

Then, in Scene3DViewController, we will setup an EAGL context, create a Skybox object and add it to the scene.


private func configureContext()
{
    self.context = EAGLContext(API: EAGLRenderingAPI.OpenGLES3)
    EAGLContext.setCurrentContext(self.context)
}

private func configureView()
{
    self.scene3DView.context = self.context

    let skybox = Skybox()
    self.scene3DView.addSceneObject(skybox)
}

The Skybox object wraps up a GLKSkyboxEffect to display any static background, just for now. (We took a set of free skybox textures from Emil Persson’s webpage)


private func configureCubemap()
{
    let fileNames = [
        NSBundle.mainBundle().pathForResource("posx", ofType: "jpg")!,
        NSBundle.mainBundle().pathForResource("negx", ofType: "jpg")!,
        NSBundle.mainBundle().pathForResource("posy", ofType: "jpg")!,
        NSBundle.mainBundle().pathForResource("negy", ofType: "jpg")!,
        NSBundle.mainBundle().pathForResource("posz", ofType: "jpg")!,
        NSBundle.mainBundle().pathForResource("negz", ofType: "jpg")!
    ]

    let options = [GLKTextureLoaderOriginBottomLeft : false]
    self.cubemap = try? GLKTextureLoader.cubeMapWithContentsOfFiles(fileNames, options: options)
    self.skyboxEffect.textureCubeMap.name = self.cubemap.name;
}

func render(camera: Camera)
{
    self.skyboxEffect.transform.projectionMatrix = camera.projection
    self.skyboxEffect.transform.modelviewMatrix = GLKMatrix4Scale(camera.view, 50.0, 50.0, 50.0)
    self.skyboxEffect.prepareToDraw()
    self.skyboxEffect.draw()
}

Flat skybox of Skansen (by Emil Perrson)

Flat skybox of Skansen (by Emil Perrson)

Screenshot from iPhone 6s simulator showing Skansen skybox

Screenshot from iPhone 6s simulator showing Skansen skybox

That’s it for a simple 3d scene. The swift project for our 3d scene can be found here: github repo.

Next we will handle a camera so that we can gaze in any direction.

  • JayF

    is it possible to view in stereoscopic?

    • Pawel

      JayF, current version of our video 360° player does not support stereoscopic mode. For stereoscopic mode you will need at least VR mode (two views – one for each eye). You may also want to play with anaglyph effect but, for this, you will need a properly prepared video (like this one: https://www.youtube.com/watch?v=s5diEJ-V78I).

      • 黄世平

        Patel,can you provide some ideas for the VR player ? thank you

        • Pawel

          黄世平, what ideas do you mean? Could you give me more details what you need?