问题描述
我是iPhone应用程序开发的新手，尽管具有在OpenGL中进行图形编程的经验.
I'm new to iPhone app developing, though have experience in graphics programming in OpenGL.
我正在创建一个iPhone应用程序，打算在AR中显示逼真的/高质量的渲染.在尝试使用这三个选项时，我仍然不确定应该继续开发以下应用程序框架中的哪个: SceneKit，RealityKit和Metal.
I'm creating an iPhone app that I intend to display realistic/high-quality renders within AR. Whilst experimenting with these 3 options, I'm still unsure which of them I should go forward with developing my app's framework around: SceneKit, RealityKit and Metal.
我已经了解到SceneKit是建立在Metal之上的，但是我不确定是否值得花时间/精力来编写任何自定义着色器，而不是使用SceneKit可以默认提供的东西.
I've read that SceneKit is built on top of Metal, but I'm not sure whether its worth the time/effort programming any custom shaders as opposed to using what SceneKit can offer by default.
关于RealityKit，我不需要它提供的任何动画或特效，只需要真实感的渲染面即可.
Regarding RealityKit, I don't need any of the animations or special effects that it offers, just the photorealistic rendering side.
此刻，我正在使用Swift编程，过去曾使用过Objective-C，但没有计划在我的应用程序中使用任何东西.
I am programming in Swift at the moment, and had used Objective-C in the past, but haven't planned to use any in my app.
在AR和高质量模型渲染中，哪三个是最适合开发的?
Which of the three is the best to develop for AR and High-Quality Model Rendering?
推荐答案
已更新:2020年8月30日.
Updated: August 30, 2020.
RealityKit是Apple渲染技术家族中最年轻的SDK.这个高级框架于2019年发布.RealityKit是为AR/VR项目制作的，具有简化的多用户体验设置，可以在iOS/macOS上使用.没有Objective-C的遗留物，RealityKit仅支持Swift，并且仅支持声明性语法(例如在SwiftUI中). RealityKit的主要优势-可以补充/更改/自定义来自Reality Composer应用程序的场景，并且可以作为ARKit的强大扩展-尽管它也可以作为独立的AR SDK发光.在RealityKit中，主要单位是实体(( ModelEntity ， AnchorEntity ， TriggerVolume ， BodyTrackedEntity ， PointLight ， SpotLight ， DirectionalLight 和 PerspectiveCamera )具有组件，并且可以是是从资源创建的，例如ModelEntity.目前，RealityKit 2.0具有四种材料:SimpleMaterial，UnlitMaterial，OcclusionMaterial和VideoMaterial.
RealityKit is the youngest SDK in Apple family of rendering technologies. This high-level framework was released in 2019. RealityKit is made for AR / VR projects, has simplified settings for multi-user experience and can be used on iOS / macOS. There's no Objective-C legacy, RealityKit supports only Swift, and rather declarative syntax (like in SwiftUI). The main advantage of RealityKit – it can complement / change / customise scenes coming from Reality Composer app and can be a powerful extension for ARKit – although it shines as a standalone AR SDK as well. In RealityKit the main units are entities (ModelEntity, AnchorEntity, TriggerVolume, BodyTrackedEntity, PointLight, SpotLight, DirectionalLight and PerspectiveCamera) that has components and can be created from resources like ModelEntity. At the moment RealityKit 2.0 has four materials: SimpleMaterial, UnlitMaterial, OcclusionMaterial and VideoMaterial.
要特别注意iOS上的阴影-配备A11芯片组的设备会产生投射(也称为深度图)阴影，但是在A12或更高版本的设备上，我们会看到光线跟踪的阴影.
Pay particular attention to shadows on iOS – devices up to A11 chipset produce projective (a.k.a. depth map) shadows, but on devices with A12 and higher we can see raytraced shadows.
示例代码:
@IBOutlet weak var arView: ARView!
let box = MeshResource.generateBox(size: 0.5)
let material = SimpleMaterial(color: .red, isMetallic: true)
let model = ModelEntity(mesh: box, materials: [material])
let anchor = AnchorEntity(world: [0, 0,-1])
anchor.addChild(model)
arView.scene.anchors.append(anchor)
RealityKit读取.usdz，.rcproject和.reality文件格式.支持资产动画，动态，PBR材料，基于HDR图像的照明和环境音频.所有场景模型必须与锚点绑定. RealityKit 2.0可与使用生成的多边形网格一起使用场景重建功能. AR Quick Look是基于RealityKit的引擎构建的.
RealityKit reads in .usdz, .rcproject and .reality file formats. Supports asset animation, dynamics, PBR materials, HDR Image Based Lighting and environmental audio. All scene models must be tethered with anchors. RealityKit 2.0 works with a polygonal mesh generated using Scene Reconstruction feature. AR Quick Look is built on RealityKit's engine.
结论:RealityKit为您提供了高质量的渲染技术和最新的AR功能.支持LiDAR扫描仪.您可以单独使用它，也可以与ARKit一起使用. RealityKit可与UIKit故事板或SwiftUI界面一起使用.并且它至少具有样板代码.例如，RealityKit为模型的碰撞.在继承上有很多组成，因此它是一个面向协议的编程(POP)框架.
Conclusion: RealityKit gives you a high-quality render technology and up-to-date AR capabilities out-of-the-box. Supports LiDAR Scanner. You can use it alone or with ARKit. RealityKit works with UIKit storyboards or with SwiftUI interfaces. And it has a minimum of a boilerplate code. For example, RealityKit has a very simple setup for models' collision. And there's composition over inheritance – so it's rather a Protocol Oriented Programming (POP) framework.
RealityKit的本机视图是ARView.
RealityKit's native view is ARView.
@available(OSX 10.15, iOS 13.0, *)
@objc open class ARView : ARViewBase
SceneKit也是一个高级框架.苹果渲染技术系列中最古老的一种.它于2012年发布.SceneKit是为VR设计的，也可以在iOS/macOS上使用.对于AR项目，您只能将其与ARKit结合使用. SceneKit支持Objective-C和Swift.在SceneKit中，主要单元是具有自己层次结构的节点( SCNNode类)，可以是 light (SCNLight)或相机(SCNCamera)或几何(SCNGeometry)或粒子系统(SCNParticleSystem)等.SceneKit的主要优点–高度可自定义，它可以在运行时更改几何形状和材质，以30到120 fps的速度渲染场景，并且具有用于粒子系统的高级设置.有Blinn，Constant，Lambert，Phong和PBR着色器. Occlusion材质也可以在SceneKit中以自定义形式提供给我们(这里没有开箱即用的材质，就像我们在RealityKit中可以找到的).如果您需要SCNScene中的视频资料，则可以实现SpriteKit的SKVideoNode.
SceneKit is a high-level framework as well. The oldest one in Apple family of rendering technologies. It was released in 2012. SceneKit was conceived for VR and can also be used on iOS / macOS. For AR projects you can use it only in conjunction with ARKit. SceneKit supports both Objective-C and Swift. In SceneKit the main unit is a node (SCNNode class) that has its own hierarchy and can be a light (SCNLight), or a camera (SCNCamera), or a geometry (SCNGeometry), or a particle system (SCNParticleSystem), etc. The main advantage of SceneKit – it's highly customisable, it can change geometry and materials at runtime, it renders a scene at 30 to 120 fps and it has an advanced setup for particle system. There are Blinn, Constant, Lambert, Phong, and PBR shaders. Occlusion material is also available for us in SceneKit but in a custom form (there's no out-of-the-box occlusion material here like we can find in RealityKit). In case you need a video material in SCNScene you may implement SpriteKit's SKVideoNode.
示例代码:
@IBOutlet weak var sceneView: SCNView!
sceneView.scene = SCNScene()
sceneView.autoenablesDefaultLighting = true
let boxNode = SCNNode()
boxNode.geometry = SCNBox(width: 0.5, height: 0.5, length: 0.5, chamferRadius: 0)
boxNode.geometry?.firstMaterial?.lightingModel = .physicallyBased
boxNode.geometry?.firstMaterial?.diffuse.contents = UIColor.red
boxNode.geometry?.firstMaterial?.metalness.contents = 1.0
sceneView.scene?.rootNode.addChildNode(boxNode)
SceneKit读取.usdz，.dae和.scn文件格式.支持嵌套资产动画，动力学，粒子，PBR材料，HDR IBL和环境音频.对于任何节点的隐式和显式变换动画，您可以使用 SCNAction ， SCNTransaction 和 CAAnimation 类.尽管SceneKit中的碰撞的设置有点复杂.
SceneKit reads in .usdz, .dae and .scn file formats. Supports nested asset animation, dynamics, particles, PBR materials, HDR IBL and environmental audio. For implicit and explicit transform animation of any node you can use SCNAction, SCNTransaction and CAAnimation classes. Though a collisions' setup in SceneKit is a little bit complicated.
结论:SceneKit为您提供了高质量的渲染技术(但首先需要设置physicallyBased着色器)，尽管对于AR项目，您只能将其与ARKit一起使用. SceneKit是高度可定制的，可以与Swift和Objective-C一起使用，它为您提供了一组来自renderer(...)实例方法. "rel =" nofollow noreferrer> ARSCNViewDelegate 协议，可让您以60 fps的速度更新AR模型和跟踪的锚点.
Conclusion: SceneKit gives you a high-quality render technology (but at first you need to setup physicallyBased shaders), although for AR projects you can use it only with ARKit. SceneKit is highly customisable and can be used with Swift and Objective-C, and it gives you a set of useful renderer(...) instance methods coming from ARSCNViewDelegate protocol that allows you update AR models and tracked anchors at 60 fps.
SceneKit的本机视图是SCNView.
SceneKit's native view is SCNView.
@available(iOS 8.0, tvOS 9.0, *)
open class SCNView : UIView, SCNSceneRenderer, SCNTechniqueSupport
@available(OSX 10.8, *)
open class SCNView : NSView, SCNSceneRenderer, SCNTechniqueSupport
确切地说，Metal不是渲染技术，而是GPU加速器. 2014年发布.这是一个低级框架. Metal在RealityKit，SceneKit，ARKit，CoreML，Vision，AVFoundation等中随处可见.Metal在一个API的支持下结合了类似于OpenGL和OpenCL的功能.
To be precise, Metal is not a rendering technology but rather the GPU accelerator. Released in 2014. It's a low-level framework. Metal is implemented everywhere – in RealityKit, SceneKit, ARKit, CoreML, Vision, AVFoundation, etc. Metal combines functions similar to OpenGL and OpenCL under the hood of just one API.
根据Apple文档:金属是一种基于C ++的编程语言，开发人员可以用来编写在GPU上执行的用于图形和通用数据并行计算的代码. .由于Metal是基于C ++的，因此开发人员会发现它熟悉且易于使用.使用Metal，图形和计算程序都可以用一种统一的语言编写，从而可以在两者之间进行更紧密的集成."
除了Metal之外，您还可以使用MetalKit模块(2015年发布)，该模块使用更少的代码来帮助更快，更轻松地构建Metal应用.它可以在标准的金属"视图中渲染图形，可以从许多来源加载纹理，并可以与模型I/O框架提供的模型有效地协同工作.
In addition to Metal, you can use MetalKit module (released in 2015) that helps build Metal apps quicker and easier, using far less code. It renders graphics in a standard Metal view, load textures from many sources, and work efficiently with models provided by Model I/O framework.
示例代码:
import MetalKit
class RedCube: Primitive {
override func buildVertices() {
vertices = [ Vertex(position: float3(-1,1,1), color: float4(1,0,0,1)),
Vertex(position: float3(-1,-1,1), color: float4(1,0,0,1)),
Vertex(position: float3(1,1,1), color: float4(1,0,0,1)),
Vertex(position: float3(1,-1,1), color: float4(1,0,0,1)),
Vertex(position: float3(-1,1,-1), color: float4(1,0,0,1)),
Vertex(position: float3(1,1,-1), color: float4(1,0,0,1)),
Vertex(position: float3(-1,-1,-1), color: float4(1,0,0,1)),
Vertex(position: float3(1,-1,-1), color: float4(1,0,0,1)) ]
indices = [ 0,1,2, 2,1,3, 5,2,3, 5,3,7, 0,2,4, 2,5,4,
0,1,4, 4,1,6, 5,4,6, 5,6,7, 3,1,6, 3,6,7 ]
}
}
...
class CubeScene: Scene {
override init(device: MTLDevice) {
super.init(device: device)
let redCube = RedCube(withDevice: device)
objects.append(redCube)
redCube.translate(direction: float3(0,0,-10))
add(child: redCube)
}
override func render(commandEncoder: MTLRenderCommandEncoder,
deltaTime: Float) {
objects.forEach { $0.rotate(angle: deltaTime,
axis: float3(1, 1,-1)) }
super.render(commandEncoder: commandEncoder,
deltaTime: deltaTime)
}
}
结论:开发人员通常使用Metal框架为具有复杂3D环境的游戏，Final Cut Pro和Nuke等视频处理应用，Maya等3D应用生成高质量的GPU渲染，或者适用于必须执行科学研究的大数据科学应用程序.考虑一下，金属中的光线跟踪比RealityKit快得多
Conclusion: Developers usually use Metal framework to generate a High-Quality GPU Rendering for games with sophisticated 3D environments, for video processing apps like Final Cut Pro and Nuke, for 3D apps like Maya, or for big data scientific apps that must perform for scientific research. Consider, raytracing in Metal is much more quicker than in RealityKit.
MetalKit的本机视图是MTKView.
MetalKit's native view is MTKView.
@available(iOS 9.0, tvOS 9.0, *)
open class MTKView : UIView, NSCoding, CALayerDelegate
@available(OSX 10.11, *)
open class MTKView : NSView, NSCoding, CALayerDelegate
ARKit 4.0内部没有任何渲染引擎.此模块仅负责高质量的 World Tracking 和 Scene Understanding (平面检测，射线投射，场景重建和光线估计).
ARKit 4.0 has no any rendering engine inside. This module is only responsible for high-quality World Tracking and Scene Understanding (plane detection, ray-casting, scene reconstruction and light estimation).
ARKit可以使用以下三种视图类型:ARSCNView，ARSKView和ARView.
Here are three types of views ARKit is able to work with: ARSCNView, ARSKView and ARView.
@available(iOS 11.0, *)
open class ARSCNView : SCNView, ARSessionProviding
@available(iOS 11.0, *)
open class ARSKView : SKView, ARSessionProviding
@available(iOS 13.0, *)
@objc open class ARView : ARViewBase
如果您需要有关ARKit及其功能的其他信息，请阅读.
If you need an additional information on ARKit and its capabilities, please read THIS POST.
SpriteKit是用于创建和渲染2D游戏和2D图形的Apple框架.于2013年发布.您可以将SpriteKit用作独立的API，也可以将其与SceneKit和ARKit一起使用.它的主要功能是能够绘制具有物理，二维文本和形状，图像和视频的精灵.在SpriteKit中，您可以使用Objective-C或Swift编写代码.
SpriteKit is Apple framework for creating and rendering 2D games and 2D graphics. Was released in 2013. You can use SpriteKit as a standalone API or use it with SceneKit and ARKit. Its main feature is the ability to draw sprites with physics, 2D text and shapes, images and video. In SpriteKit you can write a code in Objective-C or in Swift.
官方文档:"SpriteKit是一个通用2D框架，它利用Metal来实现高性能渲染，同时提供了简单的编程界面，可以轻松地创建游戏和其他图形密集型应用.使用丰富的动画和物理行为集，您可以快速添加视觉元素的生命，并在屏幕之间进行优雅过渡." .
SpriteKit与从UIView和NSView继承的两种本机视图类型一起使用:
SpriteKit works with two native types of view that inherit from UIView and NSView:
@available(iOS 7.0, tvOS 9.0, *)
open class SKView : UIView
@available(OSX 10.9, *)
open class SKView : NSView
这篇关于高质量渲染– RealityKit vs SceneKit vs Metal的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！