Category Archives: Towards A New Cinematography

A (VR/AR/Haptics+)-based book proposal for agents-like-Lauren-Appleton

Hello literary-agents-like-Lauren-Appleton!

I hope all is well on your side of the interwebtubes! My name is Stephen Black and I have a book proposal that I believe will be of interest to you, one of the publishing companies whom you represent and millions of people interested in visual art, gaming and film-making.

Cinema 8.0 will be a timeless bestselling classic like Towards an Architecture

Towards an Architecture was written during a transitional period in which concrete, preformed materials and Modernism were replacing past paradigms and materials. Cinema 8.0 is being written during a transitional period in which VR, AR, social media and haptics are exploding the concept of "motion pictures".

An artist/producer/writer, I believe my finished text will be factual, inspirational and light-hearted. Just as Corbusier's book is evergreen, so will be mine. (A valuable resource has been this post by Ryan Holiday about the making of Perennial Seller)

Beach Road 360VR film featured at festivals in Singapore, Brisbane and Las Vegas.

SPOKEN Unity-based virtual exhibition, curated by myself, co-produced with Eugene Soh.

Captioned virtual reality scene with two real men in it

Eugene Soh and Stephen Black

i ate tiong bahru A bestseller in Singapore. With. No. Marketing.

I've long lived in Asia, have worked for CNN, Cartoon Network, Fuji TV: writer, director, cameraman, producer. Collected as an artist, published as a photographer.

Here are blog posts that exemplify some of the research that I have done. The theoretical/inspirational components, as well as a chapter outline, can be viewed upon request.

Thank you for your time and consideration literary-agents-like-Lauren-Appleton!

onwARd,

Stephen Black

public section of Engadget Experience 100K grant proposal

Bubiko Foodtour: the world's first AR superstar

I discovered information about Lauren Appleton, and her request for manuscripts, here. I wrote the above with a great deal of respect and, hopefully, in a way that prevents internet search complications or misrepresentations.

Oh yeah...a book cover about immersive environments A working title, now scrapped.

Haptics: Giving VR the Finger (1 of a series )

Rethink the body for a moment. The skin; the largest organ. The wondrous organs and processes within this sack of flesh, especially the skeletal, muscular and nervous systems. With everything working together, we can do things like jump, speak and button our clothes.

Zooming in on the act of buttoning a shirt, we see a number of interactions. The brain controls muscles related to the eyes, arms and hands. Information about texture and positioning is exchanged between brain and fingers. Muscles and tendons make movements, some obvious, some extremely subtle. Fingertips maneuver the object. Voila! The shirt is buttoned. All parts of the buttoning operation are important. However, from the viewpoint of haptics, the three nerves connected to the fingers deserve a closer look.

The median nerve controls the thumb and wrist. It also gives feeling to the skin around the palm, thumb, middle and index fingers. It runs down the inside of the arm, crosses the front of the elbow and passes through the wrist bones and connective tissue of the carpal tunnel. Compressing the median nerve over a long period can result in carpal tunnel syndrome.

Another nerve, the ulnar nerve, is connected to the muscles that bend the wrist and fingers. It passes behind the elbow. When someone says, “I hit my funny bone”, the correct reply is,”what you are actually saying is that you’ve unexpectedly stimulated your ulnar nerve, resulting in that unusual, wacky tingling sensation”.

Spock rockin' that ulnar nerve/ flexor digitalis superficialis action.

Finally, the radial nerve, which straightens the wrist, thumb and fingers. It gives feeling to the skin on the back of the hand, as well as the index finger, middle finger, and half of the ring finger. Considered by many to be the second most important nerve in the body, the radial nerve is also responsible for bending the elbow.

My interest in haptics is not yet linked to any projects, commercial or artistic. This series of haptics-related blog posts is open-ended, but, the following topics seem likely to follow, in this order: nerve endings, sensors, and haptic-related products. I am especially interested in the intersection of haptics and music. I hope to also again speak with Stelarc about his possible haptics-related projects. I was fortunate to document some of his third hand projects, as well as chat with his prosthetic head. FWIW, I have only knowingly photobombed once:

Stelarc third ear

Stephen Black's first and only photobomb.Stelarc, Uerich Lau, Steve Dixon and ? LASALLE College of the Arts

Black and white photo courtesy of Toyo Tsuchiya, from his No Se No documentation project.

Part two of this series is here.

Virtual Reality for Artists Workshop/Michael Naimark/Objectifs

I must thank Objectifs and Michael Naimark. Wow! For the past few years I have been taking notes and making blog posts for a book about cinematography in the age of VR, AR, AI etc.  Suddenly, I learn about Virtual Reality for Artists, a workshop organized by Objectifs. The workshop is by Michael Naimark, whom I know nothing about. I do sign up immediately, however, as he was Google's first resident artist.

I walk in early.Michael is writing something on the whiteboard: https://medium.com/@michaelnaimark/vr-cinematography-studies-for-google-8a2681317b3#.ra9zaxwfd

WOW!

I introduce myself and make it very clear that I am working on a book about  VR cinematography and had no idea of his work. "Not here to steal your fire and will credit you always". Michael, from the "artist as bridge school" seems fine with the situation. ((Note: That link above, in which Michael explains  a VR test he did, is an absolute must for anyone serious about VR.)

So, without further ado, I present my notes. They are in no particular order, a fact that becomes immediately obvious.

Transmedia

Nodal point

http://thomas-schwenger.de/index.php?ch=kh&sub=sub_tt&pg=npe http://www.vrphotography.com/data/pages/techtutorials/technotes/nodalptalign-tn.html Matterport http://www.forbes.com/sites/robertadams/2016/09/07/virtual-reality-is-about-to-revolutionize-these-three-industries/#3ae6fc4a7228

"So much is undefined"

"First word art, last word art"

Branching movies

interactive movies intimate space Camera based reality

Camera based reality + modeling

The Rocky of VR

Daydream........................... point of view

Modal VR

Tango

https://www.cnet.com/news/googles-tango-phone-a-preview-of-2017-augmented-reality/ http://www.theverge.com/2016/5/12/11663654/google-project-tango-expansion-vr-indoor-mapping-android Nolan Bushell on the future of storytelling https://www.engadget.com/2016/10/12/atari-co-founder-launches-modal-vr/

Kevin Kelley

poverty porn

The Czech? interactive movie made by changing lens caps on projectors. scenes of same duration,

https://books.google.com.sg/books?id=zVUEAAAAMBAJ&pg=PA31&dq=Kinoautomat+Raduz&hl=en&sa=X&ved=0ahUKEwju7L2W_PDQAhXIOI8KHdgpC8EQ6AEIKzAD#v=onepage&q=Kinoautomat%20Raduz&f=false

bullet screen

Driving While Black

The realities of arcades.

User centered design

Imax VR

Ghostbusters

https://en.wikipedia.org/wiki/Michael_Naimark

IMU first person POV

VR journalism

directed attention 65 %

SNAP vs. Google Glass

Camera that Changed the World

Pan, tilt and roll-camera directions

hyperlapse- take out the wobble

Hollywood vs. Silicon Valley

Facebook VR camera stabilization

Magic Leap

Hololens

size of display is arbitrary

Action cams

Anything where hands are involved

The feeling of being there

Robert Flaherty Documentary

Richard Deskin

Occlusion missing data

bulls eye

steadycam artists

you don't have to fill the whole 360

vr. cinematography studies  #3 you should see the whites of her eyes

ears are easy to fool. eyes are hard to fool

the content's relationship to the medium

a mixing board, to "duplicate" reality

flight simulator

multiple viewers in a shared space

VR..the intimate space, right in front of you Gamers know this

Cineoramas 1900 Paris Exposition

Representing people in VR

Equirectangular sections; the map makers' problem

orthoscopic: everything is is scale

no zooming

first person point of view

walking together

How the West Was Won

intimate language

nadir hole

ambisonic sound

Riftcam http://riftcam.com/#why180 Google Killed a Donkey binaural microphone

Jean Ropuch

Richard Leacock

Cannot look around; cannot sway. Navigation

Manipulation

Cinematic branching

https://books.google.com.sg/books?id=hRtBAQAAQBAJ&pg=PT77&lpg=PT77&dq=cinematic+branching&source=bl&ots=yuz5O7_Q6T&sig=GrJaO7CvY_kh_5D4kMgkF8b8PrQ&hl=en&sa=X&ved=0ahUKEwjJnd-v-_DQAhUSSY8KHYvRBfQ4FBDoAQgwMAQ#v=onepage&q=cinematic%20branching&f=false http://www.gamasutra.com/blogs/AlexanderFreed/20140909/225281/Branching_Conversation_Systems_and_the_Working_Writer_Part_2_Design_Considerations.php ears easy to fool; eyes difficult Abel Gance 1927 Napoleon Jeff Lynch   ???? Matterport    

Riffs on Vive/AsiaVR’s VR Development Bootcamp at DigiPen (part 2)

Part  1 of this post is here, and includes the list of presenters as well as links to their companies, topics, etc.

If you are working with VR, I would also like to let you know about a book I have been working on: you can be part of it!

Thanks again to Vive and AsiaVR for putting together such an inspiring event!

OK, again, these notes are in no particular, and in some cases I am just sharing fragmented thoughts and or links.

The Nvidia Funhouse was introduced by  Delia Hou, nVidia , in her presentation entitled NVIDIA  VR Works: Accelerating and Enhancing VR Experiences. I was captivated and took few notes. This article from GEforce, however, summarizes what Funhouse is and talks about the back end.

A brief description of Funhouse, from Youtube:

NVIDIA VR Funhouse highlights what NVIDIA Pascal-based GeForce GTX 1080 GPUs can do for VR. Built using Epic's UE4, VR Funhouse brings a new level of immersion to VR by enhancing what you see, hear and touch through a combination of great graphics, fully interactive audio and simulated physics.(the red coloring is my doing; I want to emphasize this...) As for the following clip, I reluctantly have to say that, as one who has produced hundreds of promos, this one falls short...)

Now...this video has got the FUNHOUSE HEAT! https://youtu.be/fUL610qR6WM

Finally, from The Verge:

Nvidia's VR Funhouse is a tech demo to test your PC's limits

Another presenter was Martin Eklund, from Space Plunge. Part of his presentation was about GPU-coding for animation and certain effects, but the highlight for me was Art Plunge, which is now in a Kickstarter  campaign.

Douglas Lee, from Fantahorn Studio (HTC Vive)shared his experiences in VR Game Development

Related articles: VR and game development is not a grocery store by Joe Radak on Medium) An Engadget article on a Breakdown on the costs of Character Development, by Jessica Conditt

Weiging, from HTC Vive, gave a presentation on the New Era of VR

Another presentation: Asset Creation Process for Front Defense. By Ethan Tsao, Fantahorn Studio

Vive X, the Accelerator Program.

Discovered this...(ultimate empathy machine?NOT! But would thrilled to be proven wrong.)

Joe McGinn, from DigiPen Institute of Technology, shared insights into  Presence in VR. "Presence" is what it is all about. Immersion. The technical side must become so good that it is invisible. One glitch, one dropped frame and the illusion vanishes. Joe's talk highlighted ways to put the "real" into virtual reality.

A TechCrunch article on Presence

I would like to deeply thank all of the presenters and the organizers. Starting from 2002 I was deeply involved with VR, gamememaking and educational uses of VR. In 2007, because of unforeseen circumstances with the company that I was employed with, as well as new opportunities, I shifted my focus to art and writing. The Kindle (a device built by a megacompany; a device dedicated to only ebooks!) was launched in 2007 and with it, my book writing began. I've since written five books, including a Singaporean bestseller. My plan was to wait until mobile had stabilized and VR  was almost ready for mass adoption..and here we are! This bootcamp brought me up to speed. Some things haven't changed:  although computational power has increased,  graphics are still a challenge. I was thrilled to discover, with an open mind, the state of physics engines and haptics. Now we have HMDs and hand controllers! WOW!.......It's 2017... I have characters, artworks and ideas for VR experiences and am more than ready to help define VR cinema...

Let's go!

SB

vr-haptic-demo        

CINEMATOGRAPHY 8.0 light and motion in the age of VR, 360, 4K, 8K, AR, AI, CG and drones

In Praise of Shadowsby Junichiro Tanizaki. Written in 1933, when traditional Japan was contemplating a future full of Western modernism. (50 pages) Towards a New Architecture, by Le Corbusier. Published in 1923, when traditional Western architecture was contemplating a future in which mass production was revolutionizing architecture and society. (320 pages) Cinematography 8.0 by Stephen Black. Created in 2017, when lens-based cinematography was contemplating 360 VR , as well as AR, AI, 4K, 8K CG, lens/computer imaging systems and drones. (280 pages) Reviewers of C8.0 include professional cinematographers, editors and theoreticians/artists such as  Stelarc. ......................
So... Cinematography 8.0... most certainly will have a theoretical and poetic component;the first section of the book. But the tools of  contemporary cinematography change quickly and are  extremely dependent upon economics, unlike pen and paper or paint and canvas.  SO.. there is a second part of the Cinematography 8.0. This second half, a compendium, may eventually become a book project unto itself. The compendium will be something like a combination of  encyclopedia and product guide. The entries will be written because of  their relevance to the title of the book OR because they have been commissioned. That is to say, I am looking forward to working with  production teams, individuals, software makers, camera companies or anyone with a product, movie, concept or software connected to the theme of C8.0.  This section will be done on a first come, first served basis and the pricing will soon be determined. The rates will be displayed as part of the crowdfunding campaign for C.80. Another way of saying this is that I will write, for hire, about anything or anyone related to VR for the second part of the C 8.0 book, the compendium. The crowdfunding campaign is expected to start soon, mid-December 2016. If you would like to get a head start on having a description written about your C8.0-related topic, please get in touch with me through this blog.  Thanks. Note that the book cover used for the header of this post is not the final version.  

Using Facebook Live

Because I will soon be doing another ebook giveaway AND a Kickstarter campaign, I wanted to test and learn about Facebook Live and see if it could be used to help with promotion. I should mention that, for the past three years, I have been writing and living in Bali, one result being that it was unnecessary to have a smartphone. Suddenly I find myself in Jakarta with an "old" Samsung. It has been the ideal opportunity to work with photography and Facebook Live. What I have learned in the past few days will somehow help the free ebook and Kickstarter campaigns. 1. My first Facebook Live experience...wow! Actually, this took place in Singapore, a couple of weeks ago, when I helped my friends at JDMIS do a test. At present, one segment has reached over 52,000 people and 11,000 people viewed at least part of it. These figures astounded me, though I was fully aware that JDMIS has a large fan base and the live broadcast was promoted by a boosted ad. I think less than 40 dollars were spent in total (there were a few segments). We live in amazing times. My Facebook Live audience has rarely been in the double digits. Understandable as I have done no preparation, my followers are not many and I am not paying to boost anything.
  1. Facebook Live from a technical aspect. For a serious shoot, the Wifi must be checked beforehand, especially if the camera moves even a few feet. A weak signal will stop the video or result in distortion. Sound and picture quality are acceptable but, like any video shoot, preparation is needed for the best audio and video.
  2. Facebook Live from a cinematographer's viewpoint. Without some sort of stabilizer, the picture will be bumpy. I need a Smoovie! Facebook Live uses a square format! There are filters, including one for black and white. 
  3. Facebook Live from a social media aspect. The Facebook Live Map is useful, but I don't think I gained any viewers because of it. As mentioned, I do not yet have a large audience base and as a result cannot expect a significant number of viewers, though a Facebook Ad campaign may change that. Unlike the Live video, those ads can be tagged.
Conclusion: Along with photography and writing, Facebook Live has been an interesting way to document my time in Jakarta. As for using Facebook Live to promote my upcoming campaigns, I did come up with the following idea during my experiments: During the campaigns, I will read sections from Bali Wave Ghost in interesting and quiet places in Bali that have good WiFi. Although the live recording may not attract large numbers unless I boost the post, I can later put the recorded video in my blog where I can label it with the appropriate keywords. The ability to get video online quickly and easily is, presently, the Facebook Live feature that I want to work with the most.

Facebook Live: What Would Andy Warhol Do?

Sleep Lonesome Cowboys Outer and Inner Space Flesh He would use his signature and technology to make money.Gallery Kia red and other colors_edited black cat wood   cat FB Live  

Notes on my First Two Facebook Live Shoots

Background I have  exhibited video art worldwide and have worked in network television (CNN, Fuji TV, France 2) as a producer, cameraman, lighting man, director and sound person. The first FB Live test was undertaken to document a presentation inside the JDMIS Jewellery School. The second test was meant to complement a  text and photographic project I am working on about the Tiong Bahru Market in Singapore. If there is interest by anyone, including websites, magazines or other bloggers, I would be happy to develop the following into a proper article.  For now, just these notes... Overall experience
  1. Shooting with a phone requires practice and experience. This is live! If your hand shakes due to fatigue, your audience will know it. The lightweight camera may even "throb" a bit because of your pulse. Moving the camera and/or your body will disguise this issue, but camera and body movements need to be choreographed...especially because of...
  2. The signal! I discovered in the midst of both shoots that the wifi signal was inconsistent. FB Live flashed a warning and in some cases, the recording was paused.  Though the viewfinder showed a clean image, on the viewer side the video was pixellated. In the case of documenting the jewellery making demo, this was undesired. In the case of the Tiong Bahru market, the FBLive footage reminded me of a video I'd done with Cat Hope. Great!
  3. Optics... The lens on the Samsung I used is wide, comparable to a 35mm lens on a fullframe digital SLR. Most, if not all of what I shot was 5 feet away or more. I physically moved the camera when I needed to change the composition.
Pluses -Getting to experiment with live broadcasting live. -No gear, just a phone! -Ability to shoot wild, and react to the presenters and the scene before me...no tripod! Let's dance! Still of Tanja JDMIS FB Live on desktop The display on FB is a treat... -Doing something memorable...I will always remember my first FB Live experiences. -Square format! -Sound and video were both pretty good quality. -Nice to see how many viewers, though I'd rather have a clean viewfinder. -shooting black and white video in a square format is refreshing Here is one piece of the JDMIS  footage. FB Live Samsung phone camera holder

The improvised rig, courtesy of the fine folks at JDMIS.

Challenges -Unusual titling mechanism...Seems like a screenshot for the title display is grabbed immediately once the recording starts. There is an ability to change this after the shoot however. -Areas with a weak signal result in stoppages, pauses or what looks like pixelation. I should have done more tests, though the three equipment tests in the same area did not show a weak signal and no pixelation occurred. -Faces... At one point I was sitting next to a man reading the newspaper. He seemed OK with me shooting and I couldn't resist recording him. I didn't get his name and assumed he was OK with me recording him. Things don't always go this way, though...what if a big, short-tempered person did not want to be recorded? What if that big, short tempered person thought I was shooting him or her--and I wasn't? -Camera battery unpredictable; felt that it would last for an hour and 15-20 minutes, based on tests. Battery was empty after about 30 minutes. -Everything is awesome! Oops.... I mean everything is auto....sound...exposure... - At one point little hearts floated cross the screen...no thanks. Also, a friend's avatar was on my screen while shooting... not needed and, because I was shooting live, there was no way to stop and see how to get it off. FB LIve TB Market with phone I'm excited!

VR: What Would Picasso Do? (notes and links)

Picasso's paintings, especially those from his Cubism period, are about perspective, or more accurately, experiencing one scene from multiple perspectives. His achievements in painting, sculpture printmaking and photography  make one wonder about what he would do with virtual reality, augmented reality or 360VR. In thinking about this, I discovered a book by Arthur I Miller, entitled Einstein, Picasso: Space, Time and the Beauty that Causes Havoc.  The New York Times features the book  and begins with the following quote:
Everything is possible, everything is realizable, in all and everywhere. —André Salmon
The following two paragraphs from Mr. Miller's book define Picasso's world and how he might have perceived the concept of space:
 Ideas were everywhere and so was the desire for change. Alongside the developments in mathematics, science and technology was the discovery of the conceptual quality of African objets d'art. All of these ideas helped Picasso to free himself from earlier modes of thinking. Everyone involved in cubism considered it a highly intellectual adventure with the specific goal of reducing forms to geometry. Picasso's exploration of space in his groundbreaking Les Demoiselles d'Avignon employed notions of four-dimensional space described to him by Maurice Princet, an insurance actuary interested in advanced mathematics and a member of la bande à Picasso.
And...
    In the intellectual atmosphere of 1905 it is not surprising that Einstein and Picasso began exploring new notions of space and time almost coincidentally. The main lesson of Einstein's 1905 relativity theory is that in thinking about these subjects, we cannot trust our senses. Picasso and Einstein believed that art and science are means for exploring worlds beyond perceptions, beyond appearances. Direct viewing deceives, as Einstein knew by 1905 in physics, and Picasso by 1907 in art. Just as relativity theory overthrew the absolute status of space and time, the cubism of Georges Braque and Picasso dethroned perspective in art.
Picasso owned still cameras and enthusiastically involved himself with movie productions of all kinds. This must have been an impressive show, curated by Picasso's friend and biographer John Richardson: http://www.gagosian.com/exhibitions/picasso--the-camera--october-28-2014
From: re:artist at https://reartiste.wordpress.com/2015/10/25/picasso-and-photography/ It has thus been established that Picasso was already using the camera in the early years of the century, actually developing his own prints, and that this practice went hand in hand with the pictorial evolution of the Cubist revolution. – Anne Baldassari, curator at the Musée Picasso, Paris Over the decades, Pablo Picasso pursued his research of creative resources of photography in various ways: he was combining photography with drawings, prints, engraving on glass. One of such examples can be Pablo Picasso’s painting of his wife, Olga Khokhlova: “Portrait of Olga in an Armchair”, oil on canvas, made in 1917. It was most likely painted from a photograph taken also in 1917. Russian ballerina Olga Khokhlova married Picasso in 1918.
Pablo Picasso, "Portrait of Olga in an Armchair"
Pablo Picasso, “Portrait of Olga in an Armchair”
Today, we have remarkable photographic records made by Brassaï and Gjon Mili of Picasso’s ephemeral work – “drawing with light”. Those “spatial sketches”, ethereal sculptures made in luminous pencil, as well as variations on the photogram technique and photo cutouts with André  Villers can be found in Picasso’s archives in Paris.
Back to virtual reality. A virtual reality project is a team effort. In the privacy of his studio, Picasso could use canvas, clay and film to create records of what he was thinking. VR usually requires a  production company. With VR, I believe that Picasso would best express himself as a set designer, as an artist of the theatre. Picasso had worked like this before: Portrait of the Artist as a Set Designer, an article in The Guardian. Imagine: Picasso + Cirque Soleil + VR... http://www.popsci.com/cirque-du-soleil-now-makes-you-part-show-in-virtual-reality Onward... SB     The image used at the top of the post is by Gjon Mili. This photo and the others used in this post are used only for educational purposes.

The Cinematography Book/Notes and a Reviewer

The Cinematography Book is not going to have that for title. I am now working on a proposal for a book on how lenses and computers are now redefining cinematography. But the title? Well. My first title was Angelic Visions: VR, AR, Drones and Cinematography's Second Life. Here is one of a few related posts, this one about Matt Seigel's 4K video class. And yes, I had the following idea for a cover, but that guy in the photo, with the word "angelic" nearby, didn't seem right... a symbol for cinematography For a book like this, it is important to have reviewers. Reviewers are not proofreaders; they are most important in the initial stages. Once I have the chapter outlines and the structures for the chapters themselves, I'll run it past the reviewers. The reviewers I have in mind are not all cinematographers. For now I am pleased to announce that Stelarc has agreed to be a reviewer. With his extra-extraordinary background as a performance artist and his recent experiences with a RED 6K cameras, 2 GH4 bodies, a slider, a lighting set, a 4K drone and a motorized 3 axis gimbal, his  input is a unique honor to have and I am very grateful.
What kind of book am I thinking of? I have ideas for sure, but the most attractive part of writing for me is the evolution; the way that new routes and ideas appear unexpectedly. This will not be a technical manual, that is for sure.
Hopefully I do not sound pretentious, but I am inspired by two books: Towards A New Architecture and In Praise of Shadows.
Le Corbusier, the visionary architect, wrote Towards A New Architecture early in the last century, at a time when architecture suddenly was faced with the possibilities of concrete, steel,  and electricity. Now, cinematography is faced with the possibilities of extremely high resolution digital recording technology, motion sensors, GPS, drones, AR, VR and more....
The other book, In Praise of Shadows, was written by Junichiro Tanizaki in 1933. It is meditative collection of essays about how we perceive the world.
However, though I will aim for the classic feel of these two texts, I am sure to also throw in stylistic influences from cyberpunk, the Marx Brothers and I Ate Tiong Bahru!