Tag Archives: AR

What is Augmented Reality?

The following video is a rehearsal. The starting point is a presentation I did in Detroit in July 2019. That presentation was about Autonomous Vehicles and Augmented Reality. Eventually I will have about eight videos based on that Detroit presentation. You can find the Powerpoint for that here.

It will be obvious that I am not the smoothest person to ever be on a stage. I am a photographer, a cinematographer, an artist and a writer. I am used to having time to think about what I am expressing. As I will soon be making presentations in Europe, I am trying to get through the learning curve of being onstage, asap.

In this presentation, I often refer to the Open Augmented Reality Cloud. If you are serious about AR, you must learn about their work, and download the free State of the Augmented Reality Cloud Report which is available on their website. (I am honored and humbled to say that I will be speaking about Bubiko at the OARC Symposium in Munich on October 16!)

The ARShow produced this informative podcast with Jan-Erik Vinje, Christine Perey, Jason Fox and Colin Steinman, the key cotributors to the OARC

Thank you for any suggestions. And your patience!

Oh yes... Bubiko's Unusual Guide to Augmented Reality is now available on Amazon.

Yes, the lighting is not great. This is a rehearsal, albeit an informative one.

bROADWAY: notes for a citywide AR experience

  • The recent release of citywide 3D mapping technologies allows AR objects to be placed permanently and accurately. Two companies to watch : Scape and 6D.Ai.

With an established background in visual arts, music, the performing arts and AR, I have been waiting for the opportunity to create very large scale AR artworks. Years ago I developed plans for an AR sound artwork for Singapore's Tiong Bahru, an estate composed of historic Art Deco buildings. In 2018, in Shenzhen, a citywide light show rekindled my interest in large scale AR projects.

https://youtu.be/aKbs1beYAJQ

This year, while preparing for a presentation in Detroit, another idea: an Augmented Reality project to be viewed, and experienced, from inside Autonomous Vehicles; bROADWAY

Although the ideas of bROADWAY could work in any city, Detroit is associated with the automobile industry, including AV. Detroit is the home of legends like Motown, Aretha Franklin, Stevie Wonder, the Jacksons, Detroit techno (Juan Atkins, Kevin Saunderson and Derrick May +), Bob Seger, Jack White, Alice Cooper and Iggy Pop. These points, plus distinguished architecture and public spaces, mean there could be no better city for the world’s first AR+AV citywide experience!

AVs as performers!

If anyone in Detroit knows how to make this project happen, please get in touch.

(And, as AVs are becoming more common worldwide, the idea could happen anywhere.)

This post provides more information on AR +AV. The notes about bROADWAY begin with slide 29.


An autonomous vehicle in Detroit, by May.
Detroit!

Stephen Black recently worked with Bubiko Foodtour to create an unusual guide to AR...

Bubiko Foodtour’s Unusual Guide to Augmented Reality

This blog post is a sample of the guide, which is available on Amazon.

the cover

INTRODUCTION

Hi.

Welcome to R3.

This is a response to an urgent need to share information.

 Influences: a. “make it fast, not perfect” 

                    b. wabi-sabi

                    c. ash from chaos 

                    d. Chungking Express 

                    e. The Zone System by Ansel Adams and Minor White

No person or company has paid me to have their name or product included in this document.

I am now notifying the companies and individuals mentioned. If your work is here, and I have not yet contacted you, I apologize. If you prefer that I do not share your work, let me know and  I will remove it immediately.

No one has received a promotional copy. If you have bought this, and we meet, I will buy you a beverage. Or two. If you bought this and it seems unlikely that we will meet, I will send you my other ebooks or find a way to make sure this purchase is something you are very happy with. Hint, hint, Bubiko and I are working on an app. (Many times in the past, I  have given out free ebooks; only to be surprised that people did not even open them; even my bestseller.)

If you use photos or text from this document, please credit accordingly:

Photo by Stephen Black from Bubiko’s Unusual Guide to AR

Text by Stephen Black from  Bubiko’s Unusual Guide to AR

Have a nice day.

onwARd,

Stephen Black

9/3/2019 4th edition

A

Anchor  The three points used to describe the location, in the real world, where an AR object has been placed. The more one understands about 3D geometry, the more one is prepared for AR. See also billboarding, geopost.

AR Cinema  Soon! http://www.blacksteps.tv/towards-an-understanding-of-ar-cinema-possibilities/

Arbicle  Any piece of bicycle equipment with AR capability.

A list of articles and links about AR, arbicles and bicycles: http://www.blacksteps.tv/arbike/

ARCore  The platform used to make apps for Android mobile devices. Released in 2018.

ARKit  Apple released ARKit in 2017; it is the platform used to make apps for iOS mobile devices.

[AR]T https://news.artnet.com/opinion/apple-augmented-reality-art-1624540

Augmented Driving  AR will increase the amount of information available to drivers, thus increasing driving skills and traffic safety, especially when combined with AI.

www.thedrive.com/tech/9548/the-biggest-opportunity-everyone-is-missing-in-self-driving-cars

https://panelpicker.sxsw.com/vote/100430

What Augmented Driving Will Feel Like, a SXSW presentation by Theo Calvin.

Avatar  A digital creation used to represent a person, possibly resembling a human, possibly not.

Augmented Reality  To ‘augment’ means to make something greater; to give it more power. The platform called ‘augmented reality’ is a network that adds digital information to real objects. Machines are needed to do this, and  to see the results. Thus, augmented reality is the real world and the digital information added to it, as well as the machines enabling us to experience both at the same time.See R3

   Digital information could be in the form of a 3D model of a real object made by a computer, like the furniture in AR apps made by Ikea, Wayfair and other companies.  Pokemon Go and Snapchat are other examples of AR. However, the digital information could also be live or pre-recorded video, music, podcasts,medical imagery, industrial blueprints, text information or many other types of content.

   A mobile phone, tablet, HMD (head mounted device) or eyewear is needed  to see/hear/feel the digital content.

   AR will change everything, even more than radio, television, computers and mobile phones.

Autonomous Vehicles  Vehicles capable of sensing their environment, making decisions and navigating without human input. AVs require the safest, most efficient AR data networks possible.

Augmented World Expo (AWE)  Over ten years old and still visionary. Recommended: https://www.youtube.com/watch?v=WX-geVGmOvY

B

Billboarding The term used to describe a common procedure when positioning models in AR. Billboard means that the front view of the 3D model faces the viewer. Most AR apps allow the user to rotate the model so that another side of the model is presented to the viewer.

Bubiko in the starting position; ie billboarded.

Babble Rabbit The AR masterpiece of 2019.

Occlusion, light estimation, voice commands and more:

Patched Reality and 6d allow you to learn three years worth of ARness, and see the future: 

                                                                            https://patchedreality.com/                    https://www.6d.ai/

Bubiko Foodtour AR’s first superstar, a character created by Stephen Black and Sayuri Okayama. Bubiko is one of the results of a two year food/AR research trip in Southeast Asia. Bubiko is a trailblazer who shares her AR experiences with the general public as well as with AR practitioners. Bubiko often forms partnerships, such as one with Green Bean Boy, a character made by Dominique Wu at Hummingbirdsday Studios http://www.hummingbirdsday.com/. Other collaborations are planned with the Dundercats, by Six Cat Studios https://www.sixcatstudios.com/journal/2018/4/24/the-dundercats, and creations by David Severn http://david-severn.com/ . The 3D version of Bubiko was created by Novaby. https://www.novaby.com/

Bubiko at the Mascot Hall of Fame

                                                                                                     

Bubiko debuted at the Tech in the Tenderloin event. http://www.blacksteps.tv/tech-in-the-tenderloin-2019-salvation-army-kroc-center-san-francisco/

C

Cannondale AR for bikes https://www.youtube.com/watch?v=-F3jXZDNj38

Charlie Fink's Metaverse - An AR Enabled Guide to AR & VR

Cloud see R3

Computer vision Computers use lenses, radar and many kinds of sensors to learn about the world. These different ways of “seeing” are often combined with Artificial Intelligence(AI). The end result is that computers recognize objects as well as the many kinds of information connected to them.

Convergence

Convergence author and Forbes columnist Charlie Fink tells the story of Augmented Reality (AR), a new technology that's already seeping into every smartphone and every workplace. AR's merger with new 5G and AI technologies will unleash a wave of innovation that will enable wearable, invisible, latency-free and ubiquitous computing. The book uses a kind of mobile AR called "marker AR" to allow readers to use their smartphone to bring pages to life, demonstrating with art and entertainment how the world, and every person, place, and thing, will be painted with data. https://www.amazon.com/Convergence-World-Will-Painted-Data/dp/0578460556/

D

Depth sensing Recording scenes in 3D dimensions. See volumetric video

Debbie Ding An artist and a lecturer in Singapore. Some of Debbie’s students’ work can be seen here: http://www.blacksteps.tv/ar-hkpolyu-followup-2-students-work/

Dr. Ludo's Events on Hi-Techs4Humans: Workshops, Seminars, Lectures, etc. (Facebook)

E

Edge computing There are advantages to processing data as close to the user as possible, especially in regards to the Internet of Things. This means, to a large extent, a decentralized system. This localized/decentralized approach is called edge computing.

https://www.networkworld.com/article/3224893/what-is-edge-computing-and-how-it-s-changing-the-network.html

End User The person who uses the app or device. Always important, but especially when the product is meant for a specific demographic, like the elderly.

Eye Tracking The measurement of eye motion and activity.

Eugene Soh The Dude. Probably the first in Asia (who isn’t a Facebook employee) to create an Instagram Filter. https://www.dude.sg/    www.gallery.sg  

F

Facebook Spark vs. Facebook Camera http://www.blacksteps.tv/bubiko-foodtour-test-facebook-spark-vs-facebook-camera/ = �U0���

Thank you for your interest. The rest of the definitions are available at Amazon, in the guide itself.

Bubiko at the Mascot Hall of Fame

Bubiko's guide to AR is now available on Amazon!

Bubiko stopped by the Mascot Hall of Fame to meet Reggy and his furtastic friends! She also made time to sample pierogis at the Pierogi Festival!

Bubiko at the Pierogi Festival

Bubiko's guide to AR is now available on Amazon!

Pierogis are Polish dumplings. Bubiko is a little chef from Thailand. Roll out the barrel and let the fun begin!

Bubiko’s Prizewinning Moments in Chicago and Whiting

Bubiko's guide to AR is now available on Amazon!

ATT. The plush but nonfunctional Magic Leap display.
This young woman WANTED to pose with Bubiko!
Cloud Gate, by Anish Kapoor, better known as The Bean.
The beat goes onwARd...
Hammond, Indiana. Muy mysterioso..
Midnight: Spanish karaoke
Trophies in the Purple Steer
Sunrise
The joys of not knowing where you are going...

Bubiko Foodtour test: Facebook Spark vs. Facebook Camera

Bubiko's guide to AR is now available on Amazon!

Novaby: 3D Model of Bubiko

Dana De Moreira S Hagan: Technical Support

Bubiko Foodtour's AR life began at the Tech in the Tenderloin event in San Francisco. Since that time, I have conducted tests like this, and this, in order to prepare for AR cinematography projects.

This post presents lessons learned with the Facebook Spark application, and compares Spark to a different AR application called Facebook Camera.

The 3D model of Bubiko is the same in both the Camera and Spark tests. The balloons were slightly modified, as one features Tech in the Tenderloin, and the other feature Reggy, the mascot of the Mascot Hall of Fame.

Color correction was made, in both cases, with the 'auto' function. Spark is brighter than Camera.

Spark allows the camera person to move around the model, as though it were a physical object. Spark also allows the size of the model to become different sizes. With Facebook Camera, the position and size of the model are locked. Only the front of the model can be seen, like a 2D image. Like a postage stamp.

In this case, Dana positioned Bubiko on the bottom right. We checked this position so that it also works in landscape mode. However, landscape mode seems to be unstable, based on my experiences. Fb Camera often enlarges and turns the background scene, like this:

Facebook Camera version (Sometimes using the landscape format works properly, sometimes it doesn't. In this case, I gave up after restarting five times.)
Spark version

Note the size of the pictures in the last two examples. Spark uses the dimensions of a mobile phone, Camera produces a more conventional image.

Finally: saving images. Spark, in my case, saved images directly into the iPad photo album.

Facebook Camera, on the other hand... Unless you actively go into the Facebook Archive feature, and turn it on, the images are only stored for 24 hours! I am hoping there is a way to retrieve some images I made, but so far have not been able to find a way to bring them back. Frustrating, as I am 99.9% sure I pressed "save" on the Facebook Camera app.

All images were created with an iPad.

Happy to reply to any questions or comments. Thanks for stopping by!

..............................................

I have to thank Dana Moreira S Hagan A LOT. For personal reasons, I unexpectedly found myself in Austin on a Tuesday, with the need to have the new version of Bubiko loaded in my tablet by 1PM the following Saturday.

On Wednesday, I went to Facebook, presented them with this letter:

Although I was told they would get back to me, they haven't. After constant searching, including meeting the very helpful PJ and Ramona at Impact Hub, I discovered Capital Factory. There were two events going on that night (Thursday), but two people, Brance Hudzietz and Erin Miller, put my request out on two different networks!

Finally, on Friday, about 6, I discovered Erin Ford at General Assembly. Thanks to Erin, about 7PM I started a phone conversation with Dana. Dana did not have the Mac-with-Mojave that was so desperately needed, but she did come through with the idea to use Facebook Camera which was an even better choice for testing.

Dana is great! https://dhagandev.github.io

Bubiko in Lawnspace screen test

Bubiko's guide to AR is now available on Amazon!

Bubiko: what is a stage?

The goal is to prepare for the possibilities of AR cinema.

This post documents a simple test. The Bubiko model used for the Tech in the Tenderloin event was used. Two locations: a garage and an open piece of land. The objective: to gain an understanding of what a “stage” can be in AR. AR is a new medium; to use the established techniques of theatre, television and movie is to fail to grasp the uniqueness of AR. Performance art and dance provide clues.

Notes: Spark used

  • Occlusion not a concern at this time
  • Ambient light a constant
  • Size and scaling of Bubiko purposely varied

Bubiko was created by Stephen Black and Sayuri Okayama

  • iPad used; no manual controls nor color correction
This is Bubiko Foodtour, still with the balloons and banner she used for her debut in San Francisco, at Tech in the Tenderloin’s 2019 TNT Tech Fest,  Bubiko is a work in progress, this is not her final version.
Bubiko is a 3D model made by Novaby, based on ideas by Stephen Black and Sayuri Okayama.
These images are a screen test of sorts, a way of documenting the possibilities and challenges of AR cinematography.
AR cinematography is one of my AR goals, especially making content for autonomous vehicles.
In these images. Bubiko was positioned near the white log. Only the camera (an iPad) moved. The software used was Spark. It was stable, except when used at long distances, in which case Bubiko appeared some distance away from the post.
The camera automatically controlled exposure. The light was even, the time being late afternoon/early twilight.
Occlusion was not important in this test, but I would love to use a software like 6D for testing.
Again, Bubiko did not move, but her size was changed in camera. AR cinema reminds me of machinima. There is another test of Bubiko HERE.
The format of the screen size is both an opportunity and a challenge.
Bubiko Foodtour models for some AR camera tests.
Interesting that the same model can have the size of a mouse, or a giant, or anything in between. Size in AR was the topic of this blog post. Thanks again to Fabian Winkler.
Bubiko is actually closer to the stump than this image suggests.
Sparks had some difficulty as I moved away from Bubiko's position. For this experiment it was simply an interesting development. If this were actually part of an movie shoot, I would reposition her, or do testing. It should be noted that wifi was not needed to create these images.

Arbicles

A list of all of my AR+bike blog posts is here.

The future of AR in the world of bicycling
A chart depicting the hypothetical uses of augmented reality in the world of bicycling.

ARBICLE- A piece of bicycle equipment, with partial or total augmented reality functionality. A bicycle light with some AR functions is an arbicle. A device which acts as a "AR computer", exclusively managing AR functions for the bike, traffic and rider is also an arbicle.

The word "arbicle" was coined by Stephen Black in 2019, in response to a perceived need for AR-compatible bicycle gear.Stephen Black is a writer, a visual artist and an AR/VR producer.

The chart above is a hypothetical overview and no such network or devices now exist. LIDAR is used an example of a traffic monitoring system. In reality, LIDAR is heavy, delicate and extremely expensive. Unlike autonomous vehicles, bicycles must have an attentive human steering and braking. Thus,LIDAR would be unnecessary. However. sensors detecting objects within a range of 50 feet around the bike would be advantageous. An arbicle featuring a rear view camera with a feed into the visor or smartglasses is an obvious need. ARbicles specializing in emergencies would be another obvious need.

This chart is a result of research and thought related to the book I am working on about AR and traffic safety, focusing on bicycles. The working title is AR on Wheels.

I will be making a presentation about AR at the Great Ohio Bicycle Adventure, in Port Clinton, on June 18. I

Also, if you are near San Francisco's Tenderloin District on June 28th, you are warmly invited to the world premier of Bubiko Foodtour!

Bubiko will be at the Novaby booth at the Tech in the Tenderloin Fair! her app will not be on dispaly, as we haven't made it yet, but Bubiko will be hosting an interactive event that is sure to be a crowdpleaser.

Grudge match: Bubiko Foodtour vs. Jackie Chan
Grudge match

Cloudy Traffic

A collection of posts regarding AR and Bicycling can be found here.

Traffic planning in the age of the cloud. I have read a lot and talked to many experts, but the questions in this little chart still are unanswered. To me, at least. Any ideas or thoughts are welcome; just use the comments section. Thank you! If you are an expert and want to be credited, or interviewed for my upcoming books just let me know. One book is about AR and bicycle safety. the other is about AR and traffic. (Have not yet finalized the titles.)

Obviously, the following is hypothetical; I can't think of any existing city which could accommodate five one-way lanes of traffic.

CURB ACTIVITY: Pedestrians. Parking spots for ebikes,bikes, scooters, fire hydrants, subterranean electricty and water lines, wheelchair access, bus stops, taxi/Uber/Lyft pick up and drop off areas

BIKE LANE: Individual and groups of riders, ll requiring two-way communication for weather, traffic patterns etc.

AV LANE small and large vehicles, some functioning as mobile rooms (classrooms with remote teachers, discussion rooms, meeting rooms, chill rooms, even mobile workout spaces with offsite trainers)

EMERGENCY LANE/DELIVERY LANE/BUS LANE Could something like this be managed? By whom? Would involve communicating with AVs.

TRAFFIC SIGNAL With proper traffic planning, autonomous vehicles would not need to "see" a physical light. and in this slightly futuristic scenario, could the traditional red/green/yellow lights be replaced by screens? Not just colors could be displayed, but traffic information.

WHO IS USING WHAT KIND OF AR?
Pedestrians- shopping, entertainment and internet services/information and maps for walking.

Bicyclists- maps and traffic information, internet services, including music.

Autonomous Vehicle 1 Kids using smartglasses to watch Bubiko Foodtour AR movies, one parent uses smartglasses to plan the day's activities, the other parent, also with smartglasses, attends an AR meeting conference call.

Autonomous Vehicle 2 A regularly scheduled "classroom mobile"with four students, a human teacher and AR educational materials.

QUESTIONS/THOUGHTS:

Vehicle 1: the Bubiko AR movie is on Google Cloud, the AR meeting is hosted on Apple's iCloud service, the other parent is using Facebook's cloud service. The windshield display is by WayRay and hosted on Microsoft's Azure cloud service.

Vehicle 2 is fitted with a Sony mobile AR classroom and the windshield AR hosting software is by Kamatera.

Emergency Vehicles: capable of seeing and interacting with emergency situations before arriving. Connected to a central traffic planning system, emergency vehicles and AVs are capable of maneuvering so that emergency vehicles get to their destination as quickly as possible.

Autonomous Delivery vehicles can be programmed to deliver with minimum traffic interference; for example at specific low traffic times.

Curb Activity is monitored by Open Curbs, Populus and Remix, all of whom feed information into private and government service providers who then use AR to provide information to those in traffic.

Road surface conditions: Potholes, floods, ice and snow. Monitored and then delivered to a central traffic planning agency.

Underground: sewers, flood control, power cables and more. Information available by AR

Drones: somehow drones will enter into the mobile AR picture...

In researching the above, I just discovered n excellent article about MDS, Mobiliy Data Specification.Here is a excellent article by David Zipper about MDS. Also highly recommended: this presentation by Regina Clewlows and Don MacKenzie.

................................

AR and bicycle interaction: a collection of posts on this topic is here.

If you happen to be in Port Clinton, for the Great Ohio Bicycle Adventure, stop by!

Bubiko's public AR debut! San Francisco! June 28! Bubiko's very, very excited! Tech in the Tenderloin

Thanks for leaving a comment!