Imagine being able to enjoy all the thrills and spills of a rollercoaster ride in a container terminal, whizzing past ship-to-shore cranes, winding through container stacks, and looping over the quay and yard equipment. While it sounds crazed, virtual reality offers such an experience. But there’s a serious side to VR as well that ports should be latching on to.
Software developer Tideworks Technology was one of a handful of specialists that offered the port VR experience at conferences this year. “Our designer had taken our Terminal View 3D rendering of a terminal facility and added some rollercoaster tracks to take you through,” says Harvey Bauer, Tideworks’ marketing & contracts director.
Putting on the Oculus Rift headset and jumping on for the ride was a bit of fun and something of an experiment, he says, but Tideworks is examining the opportunities that VR could deliver in its specialist field of terminal operating systems.
“We are investigating this technology and its potential applications in the operational context, as a TOS provider,” he says. “One example could be as a training tool; rather than putting a new superintendent into the actual terminal environment, the person could explore the terminal and figure out operational details without going into the real world, where there are risks and dangers.”
Test bed
Simulators and visualisation environments offer a powerful option for testing and understanding the functionality of complex multi-technical systems and for terminal layouts and designs, says Hannu Santahuhta, who leads Kalmar’s simulator and visualisation development projects. Importantly, visualisation adds credibility, he says.
Jyrki Luukkonen, from Kalmar’s terminal development team, says: “Simulators are not a revolution in the industry – what is changing now is the visualisation part.”
Where previously Kalmar made simple videos to demonstrate how a new terminal design would operate, it is now increasing its focus on VR and visualisation, using Oculus Rift technology. “The results have been extremely promising and we see new opportunities opening up,” says Mr Luukkonen.
“We are engineers – we want to see the numbers, but our customers want to visualise things. The benefit of virtual reality is that the experienced eye – i.e. the port operator – can easily spot something too close to something else, a corner that isn’t safe, the hot spots, where driver congestion might build up, and what we should move or change in order to keep up productivity. These things become evident when you have a realistic, correctly dimensioned 3D model – in ‘reality’ it is very easy to see immediately if something is wrong. This is for the chief executive and senior management to see, not necessarily what the engineer needs to see.”
Drawing a line
Where does the computer game end and the serious virtual reality begin? The aim is to get the ‘virtual’ as ‘real’ as possible, he says. That means a model that won’t allow you to drive terminal handling equipment at 100 mph through the container stacks.
“There is no typical task with this technology – each case is different. It could be that the customer is looking to optimise equipment usage in his yard. The ship-to-shore cranes are fixed, they know what kind of ship loading/discharging there is and they know there are delays for truck drivers coming into the terminal. They want to find a solution. The simulator will build a mathematical formula based on how the operation is running now and finding that optimal solution – how many pieces of container handling equipment should be used, what could be the bottlenecks, what are the consequences if you increase the equipment, perhaps getting more and more productivity up to a point where productivity then drops off because of traffic congestion. The simulator will give the theoretical answer as to how much equipment I should allocate for landside truck handling to maintain a maximum 30-minute gate-in to gate-out time.
“VR, via large-screen or Oculus Rift, adds credibility to the simulator. You can see that the straddle carrier isn’t taking four containers at a time and the operation is safe.”
Kalmar is another operator that has embraced VR in its marketing: at TOC Europe visitors could ‘walk’ on the service platform of its latest automatic stacking cranes and look at the solutions developed by Kalmar engineers.
“This shows that virtual reality is useful in terminal design, in machinery design and in marketing too,” says Mr Santahuhta. “For example, it is an absolutely great tool for checking visibility from the cabin of a piece of equipment. For the service operator being trained with that new set-up, we can make it possible for a trainee or newcomer to ‘climb’ on to a virtual crane and get familiarised on the gates to open, what needs to be fixed or aligned and what must not be changed.”
Next generation
For a generation brought up on computer games, this type of training will seem perfectly natural, points out Mr Luukkonen. “The machines are developing too, of course, and the amount of information available is increasing fast. The challenge will be how to capture, manage and utilise that information and add it to virtual reality. This new technology is an open door – any number of utilisations might be possible. We can’t say today what it might make possible but I am very excited about the future.”
The Port of Bristol took visitors on a virtual tour at Multimodal in Birmingham in April. Using an Oculus Rift headset, visitors were able to experience interactive, 360-degree footage of activity at Bristol, ‘revealing the inner workings of this innovative port’.
Director of communications Sue Turner said: “We wanted to get away from the passive, dull films you always see at exhibitions and come up with a totally new way of showing people what we do. This port tour is interactive and the level of immersion really puts you in the driving sea – or, in this case, on the bridge of a ship, or high on a gantry crane.”
The 3D film was made by Bristol-based Lobster Pictures, whose managing director, Robbie Allen, says: “We learned some interesting things from Multimodal – especially how much you want to immerse people. Not everyone likes having the Oculus Rift stuck to their face with three straps. In fact, the Google cardboard VR viewer makes a great alternative, which you can take away from your face quickly. The technology from that point of view is very much in its infancy; in five years’ time, I believe it will be a lightweight headset.”
Lobster filmed for the ‘wow factor’, he said – attaching cameras to a crane and a forklift and filming from a helicopter too. “We can put a camera system in places where it’s too hazardous for anyone to go – for example, on a container being moved by a container crane, giving a view you would never otherwise have. The challenge is synchronising and aligning six video streams – there’s a lot of post-production work involved.”
Going beyond this, it’s perfectly possible to integrate virtual or planned features into the 3D film, he says. “So we could add a new terminal, for example, and you could easily see how it would look from the top of a crane, or a ship or another perspective.”
Real time VR broadcasting is the future
Fast forward ten years and how might virtual reality serve the ports sector then?
Robbie Allen of Lobster Pictures says: “A big part of it for us, and something we are actively developing, is getting that 3D environment in real time to someone’s headset. We have built a system where there’s a set of cameras in one room and someone in a completely different place can put on a VR headset and they will be there, with 360-degree vision.
“The computing power today is quite significant – it is only a matter of time and we will be able to take all the video streams and do live stitching. The implications of that are quite amazing. It is like broadcasting. As computers get faster and technology comes down in price, that is the future of broadcasting.”
A large part of Lobster’s business expertise is in long-term time lapse photography for big construction and engineering projects, together with the ability to add in 3D for mock-ups so clients can see what a building or development will look like in a year.
An early project involved a pipeline being built across the Emirates; instead of having to travel to or be based in portable buildings in the desert, the project director was able to watch and monitor progress at four different sites, from his office in Abu Dhabi.
“Another practical application would be where we normally send people into areas with hazard chemicals, etc., wearing protective suits and equipment; instead of sending in a person, you could see what a robot sees in real time.”
