深掘りARKit

深掘りARKit with Unity
⾕⼝直嗣

では早速起動してみましょう！

ざっくりアプリ構成イメージ

UnityARSessionNativeInterface
C# on Mono
iOS Native

UnityARSessionNativeInterface.cs
でNativeの関数をラップ

namespace UnityEngine.XR.iOS { 
………………………
 
[DllImport("__Internal")] 
private static extern void session_SetPlaneAnchorCallbacks(IntPtr nativeSession,
internal_ARAnchorAdded anchorAddedCallback,  
internal_ARAnchorUpdated anchorUpdatedCallback,  
internal_ARAnchorRemoved anchorRemovedCallback); 
 
Unityのエンジンの中に統合されているっぽい
こんな感じでNativeのARKitを呼び出す

ざっくり動作イメージ

UnityARSessionInterface
3D描画
空間の把握

Camera関連
Unityのカメラの位置、画⾓、
クリッピングをiPhoneのカメラと
合わせる

ARKit
Unity
AR空間の
カメラの位置
の推定
AR空間の
平⾯の位置
の推定
iPhoneの
カメラに
合わせた画⾓
AR空間の
平⾯の
HitTest
AR空間の
明るさ
UnityARSessionInterface
Unityの
カメラに位置
をセット
影
Occlusion
AR⽤背景
明るさに応じ
たライト
ヒットテスト

UnityARCameraManager.cs
UnityARCameraNearFar.cs

void Update () { 
 
if (m_camera != null) 
{ 
// JUST WORKS! 
Matrix4x4 matrix = m_session.GetCameraPose();
m_camera.transform.localPosition = UnityARMatrixOps.GetPosition(matrix);
m_camera.transform.localRotation = UnityARMatrixOps.GetRotation (matrix); 
 
m_camera.projectionMatrix = m_session.GetCameraProjection (); 
} 
 
} 
 
ARKitからカメラの位置情報を取得してUnityのカメラにセット
UnityARCameraManager.cs
ARKitからカメラのProjectionMatrixを取得してUnityのカメラにセット

void UpdateCameraClipPlanes() 
{ 
currentNearZ = attachedCamera.nearClipPlane; 
currentFarZ = attachedCamera.farClipPlane; 
UnityARSessionNativeInterface.GetARSessionNativeInterface ().SetCameraClipPlanes (currentNearZ,
currentFarZ); 
} 
 
// Update is called once per frame 
void Update () { 
if (currentNearZ != attachedCamera.nearClipPlane || currentFarZ != attachedCamera.farClipPlane)
{ 
UpdateCameraClipPlanes (); 
} 
} 
 
UnityのカメラのNear, Farを取得してARKitのカメラにセット
UnityARCameraNearFar.cs

背景
iPhoneのカメラの
ビデオストリームを背景に

ビデオストリームでカラーバッファをクリア
それから3D描画

UnityARVideo.cs

void InitializeCommandBuffer() 
{ 
m_VideoCommandBuffer = new CommandBuffer();  
m_VideoCommandBuffer.Blit(null, BuiltinRenderTextureType.CurrentActive, m_ClearMaterial); 
GetComponent<Camera>().AddCommandBuffer(CameraEvent.BeforeForwardOpaque,
m_VideoCommandBuffer); 
bCommandBufferInitialized = true; 
 
} 
UnityのCommandBuﬀerの機能を使って描画コマンドを挿⼊
UnityARVideo.cs

CameraEvent.BeforeForwardOpaque

public void OnPreRender() 
{ 
……… 
// Texture Y 
_videoTextureY = Texture2D.CreateExternalTexture(currentResolution.width,
currentResolution.height, 
TextureFormat.R8, false, false, (System.IntPtr)handles.textureY); 
_videoTextureY.filterMode = FilterMode.Bilinear; 
_videoTextureY.wrapMode = TextureWrapMode.Repeat; 
_videoTextureY.UpdateExternalTexture(handles.textureY); 
 
// Texture CbCr 
_videoTextureCbCr = Texture2D.CreateExternalTexture(currentResolution.width,
currentResolution.height, 
TextureFormat.RG16, false, false, (System.IntPtr)handles.textureCbCr); 
_videoTextureCbCr.filterMode = FilterMode.Bilinear; 
_videoTextureCbCr.wrapMode = TextureWrapMode.Repeat; 
_videoTextureCbCr.UpdateExternalTexture(handles.textureCbCr); 
 
m_ClearMaterial.SetTexture("_textureY", _videoTextureY); 
m_ClearMaterial.SetTexture("_textureCbCr", _videoTextureCbCr); 
…… 
ビデオストリームをテクスチャーとしてセット
UnityARVideo.cs

Anchor
ARKitで認識した平⾯

UnityARGeneratePlane.cs
UnityARAnchorManager.cs

namespace UnityEngine.XR.iOS 
{ 
public class UnityARGeneratePlane : MonoBehaviour 
{ 
public GameObject planePrefab; 
private UnityARAnchorManager unityARAnchorManager; 
 
// Use this for initialization 
void Start () { 
unityARAnchorManager = new UnityARAnchorManager(); 
UnityARUtility.InitializePlanePrefab (planePrefab); 
} 
 
void OnDestroy() 
{ 
unityARAnchorManager.Destroy (); 
} 
 
void OnGUI() 
{ 
List<ARPlaneAnchorGameObject> arpags = unityARAnchorManager.GetCurrentPlaneAnchors (); 
if (arpags.Count >= 1) { 
//ARPlaneAnchor ap = arpags [0].planeAnchor; 
//GUI.Box (new Rect (100, 100, 800, 60), string.Format ("Center: x:{0}, y:{1}, z:
{2}", ap.center.x, ap.center.y, ap.center.z)); 
//GUI.Box(new Rect(100, 200, 800, 60), string.Format ("Extent: x:{0}, y:{1}, z:
{2}", ap.extent.x, ap.extent.y, ap.extent.z)); 
} 
UnityARAnchorManagerを作る
UnityARGeneratePlane.cs
位置検出したら返ってくる

Hit Test
画⾯タップした位置の
平⾯の座標をゲット

namespace UnityEngine.XR.iOS 
{ 
public class UnityARHitTestExample : MonoBehaviour 
{ 
public Transform m_HitTransform; 
 
bool HitTestWithResultType (ARPoint point, ARHitTestResultType resultTypes) 
{ 
List<ARHitTestResult> hitResults =
UnityARSessionNativeInterface.GetARSessionNativeInterface ().HitTest (point, resultTypes); 
if (hitResults.Count > 0) { 
foreach (var hitResult in hitResults) { 
Debug.Log ("Got hit!"); 
m_HitTransform.position = UnityARMatrixOps.GetPosition
(hitResult.worldTransform); 
m_HitTransform.rotation = UnityARMatrixOps.GetRotation
(hitResult.worldTransform); 
Debug.Log (string.Format ("x:{ 0:0.######} y:{ 1:0.######} z:{ 2:0.######}",
m_HitTransform.position.x, m_HitTransform.position.y, m_HitTransform.position.z)); 
return true; 
} 
} 
return false; 
} 
iPhoneの画⾯上の点を指定してARKitのヒット
テストを呼ぶ
UnityARHitTestExample.cs
Hit Testの結果をチェック

影
AR空間で影を落とす

https://www.youtube.com/watch?v=v3Gym4mlcn0

struct v2f 
{ 
float4 pos : SV_POSITION; 
 
LIGHTING_COORDS(0,1) 
}; 
 
 
v2f vert(appdata_base v) { 
v2f o; 
o.pos = UnityObjectToClipPos (v.vertex); 
 
// 5.) The TRANSFER_VERTEX_TO_FRAGMENT macro populates the chosen LIGHTING_COORDS
in the v2f structure 
// with appropriate values to sample from the shadow/lighting map 
TRANSFER_VERTEX_TO_FRAGMENT(o); 
 
return o; 
} 
 
fixed4 frag(v2f i) : COLOR { 
 
// 6.) The LIGHT_ATTENUATION samples the shadowmap (using the coordinates
calculated by TRANSFER_VERTEX_TO_FRAGMENT 
// and stored in the structure defined by LIGHTING_COORDS), and returns the value
as a float. 
float attenuation = LIGHT_ATTENUATION(i); 
return fixed4(1.0,1.0,1.0,1.0) * attenuation; 
} 
ShadowMapを取得
MobileARShadow.shader

Occlusion
実物の後ろにあるものは描画しない

https://www.youtube.com/watch?v=xEmLGoVs7Fs

机の上は描画されている
机の下は描画されて
いない

ZWrite On 
ZTest LEqual 
ColorMask 0 
 
CGPROGRAM 
#pragma vertex vert 
#pragma fragment frag 
 
#include "UnityCG.cginc" 
 
struct appdata 
{ 
float4 vertex : POSITION; 
} ; 
 
struct v2f 
{ 
float4 position : SV_POSITION; 
} ; 
 
v2f vert (appdata input) 
{ 
v2f output; 
 
output.position = UnityObjectToClipPos(input.vertex); 
return output; 
} 
 
fixed4 frag (v2f input) : SV_Target 
{ 
return fixed4(0.5, 0.3, 0.0, 1.0); 
} 
Zは描画
MobileOcclusion.shader
Colorは更新しない

現実世界の明るさを反映

https://www.youtube.com/watch?v=7Kk6iVr5ULo

public void Start() 
{ 
l = GetComponent<Light>(); 
UnityARSessionNativeInterface.ARFrameUpdatedEvent += UpdateLightEstimation; 
} 
 
void UpdateLightEstimation(UnityARCamera camera) 
{ 
// Convert ARKit intensity to Unity intensity 
// ARKit ambient intensity ranges 0-2000 
// Unity ambient intensity ranges 0-8 (for over-bright lights) 
float newai = camera.lightEstimation.ambientIntensity; 
l.intensity = newai / 1000.0f; 
 
//Unity Light has functionality to filter the light color to correct temperature 
//https://docs.unity3d.com/ScriptReference/Light-colorTemperature.html 
l.colorTemperature = camera.lightEstimation.ambientColorTemperature; 
} 
 
イベントを登録
UnityARKitAmbient.cs
ライトの⾊温度を変更

垂直な⾯をUnity上で作って
物理シミュレーションと
Occlusion

https://www.youtube.com/watch?v=akCCwPeHF9k

GPSの情報と組み合わせ

https://www.youtube.com/watch?v=CGJHELNepAI

深掘りARKit

Recommended

Recommended

More Related Content

What's hot

What's hot (14)

Similar to 深掘りARKit

Similar to 深掘りARKit (20)

More from Naoji Taniguchi

More from Naoji Taniguchi (20)

深掘りARKit