Category Archives: AR

Autonomous Vehicles and Augmented Reality (1/23/20 Collider presentation follow up)

stephen black will be giving presentations in Toledo OH, MIT and elsewhere. Details here.

Facial scan of Stephen Black by Chiaki Williams, using Scandy + iPhone 11 Pro Max

A TEAM OF HACKERS, MINUTES AFTER A MASSIVE EARTHQUAKE. Their mission: study car navigation usage to identify surviving roads, and the traffic upon them. Location: Tokyo, immediately after the 2011 Tohoku Earthquake and Tsunami. The team: members from Toyota, Honda and Nissan; the first time in history the car manufacturers worked together.

This was one of the pieces of information shared by an attendee of my presentation about Autonomous Vehicles and AR. The point of the Japanese automaker story was to illustrate the powerful advantages of interoperable standards. For AR to reach its full potential, AR service providers like Microsoft, Amazon, Apple and others must agree upon standards that are interoperable and respectful of privacy. In other words, a GeoPose standard must be established.

GeoPose is a term given to a 6DoF geospatial reference or link of  an object (physical or digital) to the real world. It can be used to anything from visual positioning systems for AR to self driving cars or flying drones. As URL are vital to the internet, GeoPose will be to AR and real-world spatial computing.

GeoPose is a part of a larger vision for an Open Spatial Computing Platform (OSCP) inspired by the web, but for the era of real-world spatial computing.

At the bottom of this post is an update from Jan-Erik Vinje, regarding the recent activities of the OGC GeoPose Standards Working Group.

Imagine an eight story industrial building which houses 20 large workshops, six of which belong to one company and two workshops are shared by all. Question: will each company have an independent AR guidance system? Will visitors have to switch AR apps when going from one workshop to another? Will inventory suppliers have to download apps for the building and for each workshop? Will repair and maintenance crews have access to one AR system that shows wiring, water, conduits, etc? In emergencies, will fire fighters, police and first responders be able to use one system that indicates where all occupants are, where all combustibles are, where the fire escapes are?

There are countless reasons why GeoPose should be standardized, but emergency responses, medical and traffic regulation are the most obvious. On my website are three posts regarding GeoPose, my ”favorite” being one on the difference between GeopPose and GPS. I am very appreciative of the time and thoughtful explanations of the Open Augmented Reality Cloud team, especially Jan-Erik Vinje, OARC’s managing director.

OARC is a volunteer-run dynamic group who are now working on how to create GeoPose standards that are open to all. If you are serious about AR, please consider joining OARC.

Kura Gallium: Thunderbirds are go!

The talk popped from topic to topic, driven by audience questions and observations, with videos providing starting points.

Hyperreality short film by Keiichi Matsuda

Navdy- Look Forward and experience the future of driving

Tencent Keen Security Lab Experimental Security Research of Tesla Autopilot

PyTorch at Tesla- Andrej Karpathy, Tesla

Transportation Paradigms at the Intersection of Post Industrial Quantum Manifestations

Cannondale Simon Service AR

Nissan Invisible-to-Visible Technology@ CES2019

Bubiko Foodtour Meets Tranzient 1.0

Civil Maps Augmented Reality Maps and Localization

A volumetric scan from the presentation. And yes, he is single. And intelligent: he described how years ago he hacked a turntable so it would rotate slowly enough for him to do a 3D scan. (The image was created in the presentation with an iPhone 11 Max Pro and Scandy software.)

Jeff Koons Art installations in AR

I hope that Dan, who shared the story about the automakers' response to the earthquake, posts the link.

I enjoyed the presentation very much. Thanks to Altimetrik and all who attended.

Next stop Toledo, then MIT.

My two part article about the future of AR on the AIXR website. 

There was one thing I forgot to mention in the presentation: Bubiko Foodtour’s Unusual Guide to Augmented Reality. ARPost wrote this about it:

There’s a great ebook out there that’ll introduce you to all the jargon you need. Whether you want to understand an article better, contribute more to a conversation, or start developing with AR technology, this is a good place to start.

The guide will be updated periodically. I have started the second edition, and after the MIT symposium I will update it again. Those who have the first edition will get a free version of the second, as well as all future updates.

Thanks to all those who have bought the first edition (please leave a review :). I am self-funded, have no sponsors and, so far, only one paid speaking engagement. Your purchase really helps.

onwARd,

SB

photo by Daniel Joseph.

Articles

Augmented Reality Is Used to Improve Autonomous Vehicle Testing Andrew Wheeler posted on December 12, 2019 | Since…www.engineering.com

https://www.thedrive.com/tech/9548/the-biggest-opportunity-everyone-is-missing-in-self-driving-cars

More information about what I have been up to in the world of AR.

re: the OGC GeoPose Standards Working Group, from Jan-Erik Vinje.

The working Group had its kick-off meeting on the 24th of January, 2020 when Christine Perey (Open AR Cloud), Jeremy Morley (British Ordnance Survey) and myself (Open AR Cloud) were elected as co-chairs. https://www.opengeospatial.org/pressroom/pressreleases/3132

The kick-off meeting had participants from across the geospatial industry, government, academia, NGO’s and large corporations like Google. It is important to note that the GeoPose standard, although an initiative from Open AR Cloud, is not being primarily going to be developed inside of Open AR Cloud, but rather in a leading standards body of the geospatial sector.

OGC announces the creation of new GeoPose Standards Working Group

opengeospatial.org

Bubiko visits Paddington

At Paddington Station, in the store filled with all that is Paddington.
The famous statue. "Please look after this Bear. Thank you."
The world famous bear meets the world famous chef. Lobby of the Paddington Hilton, London.
Bubiko, in awe.

VIEW 2019 part 4

The Hugo Guerra of Hugo's Desk. Hugo is an extremely good compositor and teacher.

From my TV and gamemaking days, I have some familiarity with compositing. Although it is almost certain that I will never become a compositor, I sat in on Hugo's seminar. I think I retained about .001% of the technical information that Hugo shared, but observing his workflow and troubleshooting tips was inspiring.

The Addams Family: Reintroducing a Classic: the presenters, co-director Conrad Vernon and Producer Alexa Schwartz, spoke about the challenges in creating a new version of The Addams Family. Although the story is contemporary, the two looked at the original drawings of Charles Addams. His drawings, featured in The New Yorker magazine, were the basis of the original TV show.

Conrad and Alexa presented like the professionals that they are. They completed each other's sentences. They kept the topics moving. Perhaps they rehearsed and planned for a long time, or maybe they just improvised so informatively because they knew their subject so well. They showed clips and explained the stories behind them. If there is an Addams Family Hall of Fame, Conrad and Alexa should be in it.

Animated movies have their own unique characteristics, but they also act as "containers" for other art forms, music being the most obvious example, drawing being another. However, successful animated characters are the result of combining drawing, sculpture, programming and... dreaming. Animated characters are often realistic, but they can ignore the laws of physics. Dreams and animation need no logic.

Having said that, the more a VFX operator/CG artist/animator studies reality, the better. Noticing the almost imperceptible muscles movements of the human body results in all kinds of possibilities, especially when one must make a character react to something unexpected. The following clip was shown by Theodore Tye, as part of his excellent presentation entitled Beyond Eye Darts: A Closer Look at Acting for Animation:

This masterclass, through the analysis of clips from animated and live- action films as well as acting theory, takes a fun, in-depth look at the study and execution of both comedy and dramatic acting for animation.

Following is one of the clips shown to exemplify a reaction worth studying.

VIEW took place in OGR Turino, an arts center in what was once a train station.
Glen McIntosh is a paleo-artist, and the supervising artist for Jurassic World.Glen's first film was Star Wars: Episode 1: The Phantom Menace. It was a pleasure to watch him work and share his stories.
Turin is a delight!
Bubiko photobombing animation bigwigs.
Hal Hickel, a legend at Industrial Light and Magic.Winner of both an Academy Award and BAFTA. He has worked on Star Wars, Jurassic Park, AI, Pirates of the Caribbean, Toy Story and The California Raisins!
Closing party. The London Siggraph people put copies of interesting black and white images on the tables, making Offtopic a bit like their Drink and Draw nights in London.

I have just settled down after travelling for over two months. I am still processing all that I learned and experienced at VIEW and an other post is on its way.

VIEW 2019 part 2

It was unexpected, suddenly having a brief discussion about cancer treatments with Professor Daniel Zajfman, the President of the Weizmann Institute of Science. Unplanned as well; Professor Zajfman was being interviewed right in front of me. What I happened to overhear was a phrase that went something like this: “Historically, cancer was perceived as one disease. Eventually, we understood that there are many kinds of cancer. Now, we realize that cancer, just like every living being, seems to be unique.”

“Wow,” I thought. I felt that I should share the two cancer treatments that I am very familiar with: the two mushroom-based treatments like agaricus blazei Murrill, and turkey tail ( https://www.youtube.com/watch?v=pXHDoROh2hA 8:50 mark), as well as the deuterium-based cancer research  of Dr. Gabor Somlyai.

At first, the presence of Professor Zajfman at VIEW was surprising. The VIEW website is a striking collection of the latest animated movies, CG creations and award-winning Hollywood technical talents. Why would a physicist whose career focuses  on atomic and molecular physics be making a presentation..He seeks to understand the astrophysical conditions found in star-forming regions, as well as working to solve the riddle of star formation. VIEW also focus on exploring the increasingly fluid boundary between real and digital worlds. Professor Zajfman listened attentively, gave me his card, and that was that. Unfortunately, I had no chance to attend his talk which was entitled What is Why and Why is it Important?

A woman warrior riding upon a flying dragon the size of a 747.

A rat who cooks in a three star restaurant in Paris. 

A mouse whose best friends are a dog and a duck.

The tortured soul of a giant robot.

A captured clownfish.

Spiderman.

Animation is the art form in which ideas like these become the foundations of global economic powerhouses.

Besides toys, the animation industry sells food products, books, health products, clothing, banking services and more. At VIEW 2019, I was very fortunate to listen to, and interact with, some of the most successful men and women in contemporary animation.

It is difficult to say who impressed me the most, but Thomas Schelesny would be in the top of the list. His presentation about the dragons created for Game of Thrones was extremely impressive.

However, after his talk he spoke to a few people, including students. He discussed the idea of being an artist. Egos were discussed. Mr. Schelesny did not mention his Emmy. What he did speak about was the professionalism needed to keep things moving. The dragons of GoT were an extremely expensive undertaking that required seamless interaction between a number of companies all over the world. These companies, for the most part, rarely work with other companies. The production schedule was dangerously short.

Mr. Schelesny exemplified the traits that are usually found in the best soldiers fighting the worst wars. He was a leader who stayed focused and thought of all members on his team.

Those watching Game of Thrones saw a beautifully choreographed and realistic battle of flying dragons. Those aware of what was happening behind the scenes , with Thomas and the Image Engine Design, saw a very different battle; one equally thrilling but real.

Bubiko learns from the master.

Bubiko meets Tranzient 1.0

I met Jim Simons, the force behind Tranzient, at the November 2019 edition of the London Augmenting Reality Meetup. He did a live demo of the Tranzient VR/music making app. The app not only performed beautifully, Jim composed some great little beats; on the spot of course.

The Tranzient app allows musicians to collaborate in VR, in real time. Tranzient is impressive on many levels. I wondered if Jim would be consider adding Bubiko to his "band".

Jim was open to the idea!

What you see above is less than an hour's work. Now that we know Bubiko is functional within Tranzient, we can literally play around.

Tranzient is highly recommended https://www.aliveintech.com/ . I just met Jim, but he is genuinely enthusiastic about making music in VR. If you are a musician and working in VR, you should definitely move on Tranzient. Jim is a bit mysterious about his musical past, but I am sure he was involved with some great projects. He sure makes Bubiko look good!

This version on Bubiko was made by Novaby, art directed by Stephen Black and Sayuri Okayama.

VIEW 2019 part 1

I wanted to meet Skarred Ghost.

Tony, Antony Vitillo, is the AR/VR consultant who runs Skarred Ghost, one of the top blogs for VR and AR. I met Tony online in 2016, and we have shared our problems and joys since then. I was invited to speak in Munich and Paris, and hoped that somehow I could finally meet Tony in person, in Europe.

However, Tony was going to be in China until he returned to Italy, to give a presentation at VIEW, in Torino, Italy. I had never heard of VIEW. When I saw it, I immediately wanted to go. But my trip was on a budget far below “low budget”. Perhaps I could volunteer, like I would do at AWE, in Munich. Tony said he would ask the organizers, and that was how we left it.

 I arrived in Turin on the morning of Sunday, October  21. I’d taken an overnight bus from Munich; the city felt both friendly and alien. Two officers in a polizia car asked me if  I knew where I was going. 

“Corso Marconi...”

Though the officers described the bus and train options very well, I didn’t pay too much attention. Bubiko and I were going to walk.

An hour after leaving the bus, this tranquil scene of the River Po.
Man Ray, Man Ray. Will have to dig up my Man Ray story. No, I never met him, but did meet Julia, his wife, twice.

And, walk we did.

The address turned out to be the administrative office for VIEW, not the venue. I was relieved when someone answered the intercom; and happy when Ricardo, a handsome young man walked out. He attentively listened to my volunteering idea and seemed genuinely interested in my Bubiko demo. Like the polizie, Ricardo gave me directions on how to the venue, a place called OGR, which seemed to be near where the bus from Munich had dropped me off. I thanked Ricardo, slung my bag over my shoulder, and began rolling my little suitcase down the road.

 A blonde young woman called out to me! She was the driver for OGR, and she was going to the venue. As she drove me to OGR, we talked. She lived outside of Turin, and told me how beautiful the golds and reds of Autumn were beneath the snow-capped peaks of the Alps.

Bamboo! Eco Hostel!
Just down the street from the VIEW administrative office.
Mustafa, whose nice little restaurant is very near to Bamboo Eco Hostel.
Bubiko with a group of very , very talented professional animators!

 We arrived at OGR, once a train station, now an art center.

I didn’t know what would happen inside, but I pretended that I did.

(to be continued)

Turino, 5AM...

PARIS/London AR Meetup @ Microsoft Reactor

This post provides notes on the above Powerpoint. The actual presentation elaborated on the points much more than I do here. Please feel free to ask questions. Also, as always I recommend joining the Open Augmented Reality Cloud. They are working to create open, interoperable standards to encourage the growth, use and understanding of AR.

And yes, another resource is Bubiko Foodtour's Unusual Guide to Augmented Reality, which is reviewed here.

  1. The following presentation uses references from the State of the AR Cloud Report published May 28, 2019 by the Open AR Cloud. All rights reserved. http://stateofthearcloud.com/ With the exception of Novaby, the 3D model making company, I have no relationships with any companies mentioned in this presentation. This research has been self-funded.

2. Stephen Black PARIS + London AR Meetup

  • SB introduction
  • Bubiko Foodtour introduction
  • Geopose

3. Nissan video/ question of geopose https://mashable.com/article/nissan-ar-virtual-reality- windshield-display/

The above link exemplifies a lot of the ideas about the future of AR in general, as well as about the potential applications of AR and AV, Autonomous Vehicles.The entire article is good, but the embedded video is excellent.

4. GEOPOSE

The photos were done as a spur of the moment test. I have done another version, featuring a 1947 Rolls Royce!

The point to be made here is that the model, Bubiko , was placed on the road ahead of the vehicle.the car, then "drove through" Bubiko, and she remained on the road behind us.

This gives us something to think about in terms of creating permanent AR experiences using geoposts.

5. Microsoft Azure Website

An example of how one company illustrates their version of the entirety of AR.

6. from a SlideShare presentation by Sergii Kozyrev

Another illustration of how the entire AR world will look like, this time based on Android.

7. OARC From State of the AR Cloud Report 2019

I cannot recommend the work of the OARC highly enough. Visit their site for the download of the first OARC Symposium Report.

8.  www.blacksteps.tv At least three posts about 'geopose'

Yes, on this blog, I presently have a few posts about 'geopose'.

9. From correspondence with Jan-Erik Vinje, Managing Director of Open Augmented Reality Cloud:

If we are successful in creating such a standard we will in effect have created the equivalent of the URL for real world spatial computing. Allowing the geopose of both real and virtual objects to be universally captured, stored, shared and understood.

This comment by Jan-Erik, and those that follow, are self-explanatory. However, one week ago, at the second OARC Symposium, the definition of 'geopost' was greatly discussed. In fact, the goal of the OARC for the next year is to fcus upon creating a final definition of 'geopost', and to work towards a working model.

10. ○ I liken it to URL because it can be seen as links between the physical world with the digital world. ○ URL links information to more information. Geopose links the world to digital information and digital information to the world.

11. ● It can even be used to create a "persistent portal" between a physical space and a virtual space. Where people from across the world who are experiencing a virtual space can go through such a portal to experience a real space (by streaming realtime reality capture data to all those in the virtual reality space).

12. ● At the same time people in the real world can walk into the virtual reality space or see virtual objects, scenes and avatars of virtual reality users projected into their physical space using AR.

13. Geopost is to AR as URL is to the internet.

14. To create a GEOPOSE 1.Physical space owner 2. AR data/ SLAM spatial location and mapping 3. AR Cloud space 4. End user(s)

This slide features a photo of the Fox Theatre in Detroit. The front of the theatre is copyrighted. Anyway planning to use the front of the Fox would need to obtain permission to do so.

This situation would be true for many buildings, copyrighted or not.

15. How to find the location? (the need for interoperability): Google maps Mapillary Military maps LADOT ebike monitoring maps(eg) Satellite-based maps monitoring floods, snow, ice, fire, crowds etc. City/State/Federal maps Company owned routing maps Bus routes Ride sharing maps Superworld Emergency routes Bike lanes Construction Traffic signals

The companies and organizations listed above would all benefit from an open AR Cloud. A comparison is sometimes made, comparing the present state of the AR Cloud with the early nonstandardized gauges of railroad tracks. Once the railroad tracks became standardized, society benefited.

Once there are interoperable standards for the AR Cloud, groups like those listed above will benefit, as will education, medicine, science and, all of society/

16. https://techcrunch.com/2017/10/08/jeff-koons-augmented-reality-snapchat-artwork-gets-vandalized Jeff Koons/Snapchat

This article shows an instance in which a geopost by Instagram was used by an artist to promote discussion about AR artin public spaces.

17. Geopose and GPS AR sculpture now being planned with Vincent Trastour/Flamingo Filters and others.

Stay tuned!

18. Bubiko and SB are always open to cross-promotions, collaborations, and exchanges!

Art, business or fun, get in touch!

19. Stephen Black LinkedIn, Facebook, twitter Bubiko Foodtour Instagram, Facebook, twitter bubikoworldtour@gmail.com

20. Thank You

(An Easter egg! If you keep clicking you will see my presentation abut AR and Autonomous vehicles.)

Scandy, New Orleans: my first volumetric camera experience

I arrived in New Orleans on the morning of October 1, 2019. The very air-conditioned overnight bus ride from Austin resulted in a bit of a raspy throat, but overall, I was excited. At 9AM, I was to meet with Charles Carriere, the President and CEO of Scandy, a company working with cutting edge 3D ideas, like scanning and volumetric video.

My first volumetric portrait experience.

Charles was friendly and informative. Our conversation was stimulating. At one point he pulled out his iPhone, waved it around me, and then pushed a button. Voila: my first portrait in 3D styleee!

My takeaway: AR will be defined by 3D objects. Yes, I knew this before, but now I have internalized that fact. Scandy's work is hugely important.

What is the difference between GeoPose and GPS?

The two previous posts on this blog are the result of trying to create a definition of GeoPose. I still have not come up with a twenty-word-or-less definition. However, the following is very helpful towards achieving that goal.

What you are about to read is a reply sent to me by Jan-Erik Vinje, the Managing Director of the Open Augmented Reality Cloud. Jan-Erik and I will both be giving presentations at the 2nd Open Augmented Reality Cloud Symposium in Munich on October 16th.

If you are serious about AR, please listen to the AR Show podcast featuring the four" workhorses" of the AR cloud.

Download the First State of the AR Cloud report as well. You can find it here.

Jan-Erik Vinje's thoughts on the differences between GPS and GeoPose:

  • GPS is a specific technology. And it is mostly used for obtaining more or less accurate geospatial positions related to a geospatial coordinate reference system. Currently an ellipsoide that approximates the earth is used to as the reference for altitude and longitude. The ellipsoid is a bit crude but it is normally less than 100 meters incorrect as a representation of where the main ocean surface is or would have been.
  • GeoPose is intended to relate to the same type of geospatial reference but adds geospatial orientation to the geospatial position. All real objects on our planet could in principle be said to have both a position and orientation. GPS provides position. If you have a stream of GPS positions you could derive a direction vector that is almost an orientation but not quite. Imagine walking down the street with a smartphone and imagine collecting a GPS location ( lat, lng, alt) every second. You can create a path through space telling you the direction you have moved with phone. What the GPS locations can not give you is the orientation you have held your device. Did you point it towards the ground, to the sky or towards a Wall across the street, and importantly how was your device rotated around that direction?
  • For that you would need another system or set of systems to provide you with your orientation. AR Cloud visual positioning systems can provide both the position and the orientation of your device. This is what we call a pose. If that position and orientation is in a geospatial frame of reference equivalent to the one used for GPS one can call the pose a geopose.
  • (SB:Hmmm that is great, but the last three lines will require a bit of serious thought...I am not 100% clear I understand. When I do, I hope to make educational drawings, or better, 3D models-in AR.)

What is Geopose? pt 2.

Jan-Erik Vinje, the Managing Director of the Open Augmented Reality Cloud, responded to my previous post about a definition of geopose. His reply is reproduced with his permission. Thanks to Jan-Erik and all of the team involved in this project. The following exemplifies all of the diligent hard work going on to create standards for AR.

Stephen Black: There is so much information here, that I need to review it carefully. It will be a wonderful challenge to come up with a definition of 'geopose' that is 20 words or less!

A reminder that Jan-Erik and I will be at the 2nd State of the Open Augmented Reality Cloud in Munich, on October 16. This appearance is part of the Greatest AR Tour in History.