Camera Framework 分析

发表于 | 分类于 | 阅读次数

Camera Framework 分析，本文主要介绍 Camera API2 相关。

类文件速查表

类文件目录

1
2
3
4
5
6
1. Framework Java API1:frameworks/base/core/java/android/hardware/Camera.java
2. Framework Java API2:frameworks/base/core/java/android/hardware/camera2
3. Framework JNI: frameworks/base/core/jni/
4. AIDL: frameworks/av/camera/aidl
5. Framework Native: frameworks/av/camera 
6. Framework Service: frameworks/av/services/camera/libcameraservice

JNI 相关

1
2
3
4
5
6
7
8
// frameworks/base/core/jni
./android_hardware_camera2_legacy_LegacyCameraDevice.cpp
./android_hardware_Camera.cpp
./android/graphics/Camera.cpp
./include/android_runtime/android_hardware_camera2_CameraMetadata.h
./android_hardware_camera2_DngCreator.cpp
./android_hardware_camera2_CameraMetadata.cpp
./android_hardware_camera2_legacy_PerfMeasurement.cpp

API 1 中，使用 jni 通过 Binder 机制和 CameraService 通信。
API 2 中，直接在 CameraManager.java 中通过 Binder 机制和 CameraService 通信。

AIDL 相关

Framework Camere AIDL 是 Camera 中客户端和服务端跨进程通信时使用的 AIDL 文件，代码都在 frameworks/av/camera/ 目录下，其中 aidl 文件一共有 16 个： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
xmt@server005:~/frameworks/av/camera/aidl/android/hardware$ tree
.
├── camera2
│   ├── CaptureRequest.aidl
│   ├── ICameraDeviceCallbacks.aidl
│   ├── ICameraDeviceUser.aidl
│   ├── impl
│   │   ├── CameraMetadataNative.aidl
│   │   └── CaptureResultExtras.aidl
│   ├── params
│   │   ├── OutputConfiguration.aidl
│   │   ├── VendorTagDescriptor.aidl
│   │   └── VendorTagDescriptorCache.aidl
│   └── utils
│       └── SubmitInfo.aidl
├── CameraInfo.aidl
├── CameraStatus.aidl
├── ICamera.aidl
├── ICameraClient.aidl
├── ICameraService.aidl
├── ICameraServiceListener.aidl
└── ICameraServiceProxy.aidl

4 directories, 16 files

frameworks/av/camera/aidl/ 目录下的 aidl 文件有两种类型：

  • 作为 Binder 中的 IInterface 跨进程通信中能提供的方法
  • 作为 Binder 中的 parcelable 跨进程通信数据传输的数据结构

很容易从名字上区分这两种类型的文件，IInterface 类型的文件都是以 I 开头的，比如：ICameraService.aidl, ICameraDeviceUser.aidl 等。不管是哪种类型的 aidl 文件，它们都会生成对应的 .java, .h, .cpp 文件，分别供 Java 层和 CPP 层调用。

IInterface 类型文件

IInterface 类型文件一共有 7 个，它们的 .java, .h, .cpp 文件，绝大部分都是自动生成的。

Java 文件是在 frameworks/base/Android.mk 中定义规则，在编译时自动生成： 

1
2
3
4
5
6
7
8
9
10
11
// frameworks/base/Android.mk
LOCAL_SRC_FILES += \
    ...
    ../av/camera/aidl/android/hardware/ICameraService.aidl \
    ../av/camera/aidl/android/hardware/ICameraServiceListener.aidl \
    ../av/camera/aidl/android/hardware/ICameraServiceProxy.aidl \
    ../av/camera/aidl/android/hardware/ICamera.aidl \
    ../av/camera/aidl/android/hardware/ICameraClient.aidl \
    ../av/camera/aidl/android/hardware/camera2/ICameraDeviceUser.aidl \
    ../av/camera/aidl/android/hardware/camera2/ICameraDeviceCallbacks.aidl \
    ...

在 out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/dotdot/ 目录下生成对应的 Java 文件： 

1
2
3
4
5
6
7
8
// out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/dotdot/
av/camera/aidl/android/hardware/ICameraService.java
av/camera/aidl/android/hardware/ICameraServiceListener.java
av/camera/aidl/android/hardware/ICameraServiceProxy.java
av/camera/aidl/android/hardware/ICamera.java
av/camera/aidl/android/hardware/ICameraClient.java
av/camera/aidl/android/hardware/camera2/ICameraDeviceUser.java
av/camera/aidl/android/hardware/camera2/ICameraDeviceCallbacks.java

.h, .cpp 文件中，ICamera.aidl, ICameraClient.aidl 两个文件是直接以代码形式手动实现的： 

1
2
3
4
5
6
7
8
9
// 1. ICameraClient.aidl
frameworks/av/camera/aidl/android/hardware/ICameraClient.aidl
frameworks/av/camera/include/camera/android/hardware/ICameraClient.h
frameworks/av/camera/ICameraClient.cpp

// 2. ICamera.aidl
frameworks/av/camera/aidl/android/hardware/ICamera.aidl
frameworks/av/camera/include/camera/android/hardware/ICamera.h
frameworks/av/camera/ICamera.cpp

其他 5 个 aidl 文件是在 frameworks/av/camera/Android.bp 中定义规则，编译时自动生成对应的 .h, .cpp 文件： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// frameworks/av/camera/Android.bp
cc_library_shared {
    name: "libcamera_client",

    aidl: {
        export_aidl_headers: true,
        local_include_dirs: ["aidl"],
        include_dirs: [
            "frameworks/native/aidl/gui",
        ],
    },

    srcs: [
        // AIDL files for camera interfaces
        // The headers for these interfaces will be 
        // available to any modules that
        // include libcamera_client, at the path "aidl/package/path/BnFoo.h"
        "aidl/android/hardware/ICameraService.aidl",
        "aidl/android/hardware/ICameraServiceListener.aidl",
        "aidl/android/hardware/ICameraServiceProxy.aidl",
        "aidl/android/hardware/camera2/ICameraDeviceCallbacks.aidl",
        "aidl/android/hardware/camera2/ICameraDeviceUser.aidl",


        // Source for camera interface parcelables, 
        // and manually-written interfaces
        "Camera.cpp",
        "CameraMetadata.cpp",
        "CameraParameters.cpp",
        ...
}

在 out/soong/.intermediates/frameworks/av/camera/libcamera_client/ 目录下生成对应的 .h, .cpp 文件，通常在该目录下会同时生成 32 和 64 位两套代码，但实际两份代码是一样的，这里选取 64 位的：

  • 64 位：android_arm64_armv8-a_shared_core
  • 32 位：android_arm_armv7-a-neon_cortex-a53_shared_core 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// 目录 out/soong/.intermediates/frameworks/av/camera/libcamera_client
// 64 位 android_arm64_armv8-a_shared_core/gen/aidl/
android/hardware/ICameraService.h
android/hardware/BnCameraService.h
frameworks/av/camera/aidl/android/hardware/ICameraService.cpp

android/hardware/ICameraServiceListener.h
android/hardware/BnCameraServiceListener.h
frameworks/av/camera/aidl/android/hardware/ICameraServiceListener.cpp

android/hardware/ICameraServiceProxy.h
android/hardware/BnCameraServiceProxy.h
frameworks/av/camera/aidl/android/hardware/ICameraServiceProxy.cpp

android/hardware/camera2/ICameraDeviceUser.h
android/hardware/camera2/BnCameraDeviceUser.h
frameworks/av/camera/aidl/android/hardware/camera2/ICameraDeviceUser.cpp

android/hardware/camera2/ICameraDeviceCallbacks.h
android/hardware/camera2/BnCameraDeviceCallbacks.h
frameworks/av/camera/aidl/android/hardware/camera2/ICameraDeviceCallbacks.cpp

parcelable 类型文件

parcelable 类型文件一共有 9 个，它们都是手动编写的代码。

Java 文件目录为 frameworks/base/core/java/android/hardware/ ： 

1
2
3
4
5
6
7
8
9
10
// frameworks/base/core/java/android/hardware/
camera2/CaptureRequest.java
camera2/impl/CameraMetadataNative.java
camera2/impl/CaptureResultExtras.java
camera2/params/OutputConfiguration.java
camera2/params/VendorTagDescriptor.java
camera2/params/VendorTagDescriptorCache.java
camera2/utils/SubmitInfo.java
CameraInfo.java
CameraStatus.java

.h, .cpp 文件并不一定是和 aidl 文件名称一一对应的，而是在 aidl 文件中定义的，比如 CameraStatus.aidl 定义如下： 

1
2
3
4
package android.hardware;

/** @hide */
parcelable CameraStatus cpp_header "camera/CameraBase.h";

parcelable 类型的 aidl 文件对应的 .h, .cpp 文件目录为 frameworks/av/camera ，对应关系整理如下： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
// .h, .cpp 文件目录 frameworks/av/camera
// CaptureRequest.aidl
include/camera/camera2/CaptureRequest.h
camera2/CaptureRequest.cpp

// CameraMetadataNative.aidl
include/camera/CameraMetadata.h
CameraMetadata.cpp

// CaptureResultExtras.aidl
include/camera/CaptureResult.h
CaptureResult.cpp

// OutputConfiguration.aidl
include/camera/camera2/OutputConfiguration.h
camera2/OutputConfiguration.cpp

// VendorTagDescriptor.aidl 和 VendorTagDescriptorCache.aidl
include/camera/VendorTagDescriptor.h
VendorTagDescriptor.cpp

// SubmitInfo.aidl
include/camera/camera2/SubmitInfo.h
camera2/SubmitInfo.cpp

// CameraInfo.aidl 和 CameraStatus.aidl
include/camera/CameraBase.h
CameraBase.cpp

ICameraService 相关

分为客户端向服务端的请求 ICameraService.aidl 和客户端监听服务端的变化 ICameraServiceListener.aidl 。这两个 AIDL 是在 CameraService.cpp 中实现对应功能的。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
// 1. ICameraService.aidl
interface ICameraService
{
    ...
    const int CAMERA_TYPE_BACKWARD_COMPATIBLE = 0;
    const int CAMERA_TYPE_ALL = 1;

    // 返回指定类型的相机设备数量
    int getNumberOfCameras(int type);
    // 根据 id 返回当前相机设备信息
    CameraInfo getCameraInfo(int cameraId);

    ...
    const int CAMERA_HAL_API_VERSION_UNSPECIFIED = -1;
    // api1 + hal1
    ICamera connect(ICameraClient client,
            int cameraId,
            String opPackageName,
            int clientUid, int clientPid);

    // api2 + hal3
    ICameraDeviceUser connectDevice(ICameraDeviceCallbacks callbacks,
            String cameraId,
            String opPackageName,
            int clientUid);

    // api1 + 指定 hal 版本（通常为 hal1）
    ICamera connectLegacy(ICameraClient client,
            int cameraId,
            int halVersion,
            String opPackageName,
            int clientUid);

    // 添加和移除 ICameraServiceListener 监听
    CameraStatus[] addListener(ICameraServiceListener listener);
    void removeListener(ICameraServiceListener listener);

    // 根据 id 返回相机支持的属性
    CameraMetadataNative getCameraCharacteristics(String cameraId);

    // 获取 vendor tag 
    VendorTagDescriptor getCameraVendorTagDescriptor();
    VendorTagDescriptorCache getCameraVendorTagCache();

    // camera api 1 获取参数信息
    String getLegacyParameters(int cameraId);

    const int API_VERSION_1 = 1;
    const int API_VERSION_2 = 2;
    // 指定 id 支持的 API 版本
    boolean supportsCameraApi(String cameraId, int apiVersion);
    // 指定 id 设置手电筒模式
    void setTorchMode(String cameraId, boolean enabled, 
        IBinder clientBinder);

    // 服务端向系统打印系统消息
    const int EVENT_NONE = 0;
    const int EVENT_USER_SWITCHED = 1;
    oneway void notifySystemEvent(int eventId, in int[] args);
}

// 2. ICameraServiceListener.aidl
interface ICameraServiceListener
{
    const int STATUS_NOT_PRESENT      = 0;
    const int STATUS_PRESENT          = 1;
    const int STATUS_ENUMERATING      = 2;
    const int STATUS_NOT_AVAILABLE    = -2;
    const int STATUS_UNKNOWN          = -1;
    // 相机设备状态变化事件
    oneway void onStatusChanged(int status, String cameraId);

    const int TORCH_STATUS_NOT_AVAILABLE = 0;
    const int TORCH_STATUS_AVAILABLE_OFF = 1;
    const int TORCH_STATUS_AVAILABLE_ON  = 2;
    const int TORCH_STATUS_UNKNOWN = -1;
    // 手电筒状态变化事件
    oneway void onTorchStatusChanged(int status, String cameraId);
}

ICameraServiceProxy.aidl 文件

CameraServiceProxy 服务是在 Java 层注册的： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
interface ICameraServiceProxy
{
    // CameraService 向代理服务发送消息，通知用户更新
    oneway void pingForUserUpdate();

    const int CAMERA_STATE_OPEN = 0;
    const int CAMERA_STATE_ACTIVE = 1;
    const int CAMERA_STATE_IDLE = 2;
    const int CAMERA_STATE_CLOSED = 3;
    const int CAMERA_FACING_BACK = 0;
    const int CAMERA_FACING_FRONT = 1;
    const int CAMERA_FACING_EXTERNAL = 2;

    // CameraService 向代理服务发送消息，通知相机设备状态更新
    oneway void notifyCameraState(String cameraId, int facing, 
            int newCameraState, String clientName);
}

ICamera 相关

Camera API1 才会使用到，分为 ICamera.aidl, ICameraClient.aidl
它们的代码是手动实现的，参考：CameraClient.h/cpp, Camera.h/cpp

ICameraDevice 相关

Camera API2 才会使用到，分为客户端向服务端的请求 ICameraDeviceUser.aidl 和服务端发给客户端的回调 ICameraDeviceCallbacks.aidl 。
表示相机设备具备的能力，能够提供的函数；这两个 AIDL 是在 CameraDeviceClient 中实现对应功能的。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
// 1. ICameraDeviceUser.aidl 
interface ICameraDeviceUser
{
    void disconnect();

    const int NO_IN_FLIGHT_REPEATING_FRAMES = -1;
    // 向设备提交捕获请求
    SubmitInfo submitRequest(in CaptureRequest request, boolean streaming);
    SubmitInfo submitRequestList(in CaptureRequest[] requestList, 
        boolean streaming);
    // 取消置顶 id 的重复请求，并返回上次请求的帧 id
    long cancelRequest(int requestId);

    const int NORMAL_MODE = 0;
    const int CONSTRAINED_HIGH_SPEED_MODE = 1;
    const int VENDOR_MODE_START = 0x8000;

    // 在流处理前执行配置请求
    void beginConfigure();
    // 根据指定输出配置，创建流
    int createStream(in OutputConfiguration outputConfiguration);
    void endConfigure(int operatingMode);
    void deleteStream(int streamId);

    // 创建输入流，返回流 id
    int createInputStream(int width, int height, int format);
    // 返回输入流的 Surface
    Surface getInputSurface();

    // Keep in sync with public API in
    // frameworks/base/core/java/android/hardware/camera2/CameraDevice.java
    const int TEMPLATE_PREVIEW = 1;
    const int TEMPLATE_STILL_CAPTURE = 2;
    const int TEMPLATE_RECORD = 3;
    const int TEMPLATE_VIDEO_SNAPSHOT = 4;
    const int TEMPLATE_ZERO_SHUTTER_LAG = 5;
    const int TEMPLATE_MANUAL = 6;
    // 根据模板创建默认请求，返回相机参数信息
    CameraMetadataNative createDefaultRequest(int templateId);
    // 获取相机参数信息
    CameraMetadataNative getCameraInfo();
    void waitUntilIdle();
    long flush();
    void prepare(int streamId);
    void tearDown(int streamId);
    void prepare2(int maxCount, int streamId);
    void finalizeOutputConfigurations(int streamId, 
        in OutputConfiguration outputConfiguration);
}

// 2. ICameraDeviceCallbacks.aidl
interface ICameraDeviceCallbacks
{
    ...
    oneway void onDeviceError(int errorCode, 
        in CaptureResultExtras resultExtras);
    oneway void onDeviceIdle();
    oneway void onCaptureStarted(in CaptureResultExtras resultExtras, 
        long timestamp);
    oneway void onResultReceived(in CameraMetadataNative result,
                                 in CaptureResultExtras resultExtras);
    oneway void onPrepared(int streamId);
    // 重复请求引起的错误回调
    oneway void onRepeatingRequestError(in long lastFrameNumber,
                                        in int repeatingRequestId);
    oneway void onRequestQueueEmpty();
}

Services 目录下的文件介绍

frameworks/av/services/camera/libcameraservice
AOSP 中这个目录下是 87 个文件，而 Qcom 的基线中增加了 27 个文件，分别为 api1/qticlient2 目录下的 25 个文件，以及 QTICamera2Client.cpp, QTICamera2Client.h 两个文件。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
.
├── Android.mk
├── api1
│   ├── client2
│   └── qticlient2
├── api2
├── CameraFlashlight.cpp
├── CameraFlashlight.h
├── CameraService.cpp
├── CameraService.h
├── common
├── device1
├── device3
├── gui
├── MODULE_LICENSE_APACHE2
├── NOTICE
├── tests
└── utils

从目录结构上可以看出，API1/2 和 HAL1/3 就是在这一层体现的。

API1/API2

APP Java 客户端调用服务端方法时，Camera API1/2 接口对应功能都是在 CameraService 中实现的，而这里的 API1/2 目录对应的就是对上层不同版本接口的处理。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
api1
├── Camera2Client.cpp
├── Camera2Client.h
├── CameraClient.cpp
├── CameraClient.h
├── client2
│   ├── CallbackProcessor.cpp
│   ├── CallbackProcessor.h
│   ├── Camera2Heap.h
│   ├── CaptureSequencer.cpp
│   ├── CaptureSequencer.h
│   ├── FrameProcessor.cpp
│   ├── FrameProcessor.h
│   ├── JpegCompressor.cpp
│   ├── JpegCompressor.h
│   ├── JpegProcessor.cpp
│   ├── JpegProcessor.h
│   ├── Parameters.cpp
│   ├── Parameters.h
│   ├── StreamingProcessor.cpp
│   ├── StreamingProcessor.h
│   ├── ZslProcessor.cpp
│   └── ZslProcessor.h
├── QTICamera2Client.cpp
├── QTICamera2Client.h
└── qticlient2
    ├── CallbackProcessor.cpp
    ├── CallbackProcessor.h
    ├── Camera2Heap.h
    ├── CaptureSequencer.cpp
    ├── CaptureSequencer.h
    ├── FrameProcessor.cpp
    ├── FrameProcessor.h
    ├── JpegCompressor.cpp
    ├── JpegCompressor.h
    ├── JpegProcessor.cpp
    ├── JpegProcessor.h
    ├── Parameters.cpp
    ├── Parameters.h
    ├── QTICaptureSequencer.cpp
    ├── QTICaptureSequencer.h
    ├── QTIFrameProcessor.cpp
    ├── QTIFrameProcessor.h
    ├── QTIParameters.cpp
    ├── QTIParameters.h
    ├── RawProcessor.cpp
    ├── RawProcessor.h
    ├── StreamingProcessor.cpp
    ├── StreamingProcessor.h
    ├── ZslProcessor.cpp
    └── ZslProcessor.h
api2
├── CameraDeviceClient.cpp
└── CameraDeviceClient.h

BasicClient 有三个重要的子类：

  • CameraClient
    如果平台仅支持 HAL 1，即 CAMERA_DEVICE_API_VERSION_1_0 ；使用 API 1/2 + HAL 1 都会对应该客户端。
  • Camera2Client
    如果平台支持 HAL 3 ，即 CAMERA_DEVICE_API_VERSION_3_0 及以上版本；使用 API 1 + HAL 3 对应的客户端。Camera2Client 会将 API1 中的接口转换为 API2 中对应的功能。
  • CameraDeviceClient
    如果平台支持 HAL 3 ，使用 API 2 + HAL 3 对应的客户端。

平台仅支持 HAL 1 时，API 2 在 openCamera 时，通过 CameraDeviceUserShim 将 API 2 转换为 API 1 ，即 HAL 1 + API 1 向下发起请求。
LegacyCameraDevice 会将 CAMERA API2 转换为 CAMERA API1 ，而 CameraDeviceUserShim 封装了 LegacyCameraDevice 。

QTICamera2Client

Qcom 的基线中增加了 27 个文件，分别为 api1/qticlient2 目录下的 25 个文件，以及 QTICamera2Client.cpp, QTICamera2Client.h 两个文件。
而 QTICamera2Client 是高通针对 API1 做的优化？在什么情况下会转换为 QTICamera2Client 呢？看如下源码： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
// 1. Camera2Client.h
class Camera2Client :
        public Camera2ClientBase<CameraService::Client>
{
#ifdef USE_QTI_CAMERA2CLIENT
friend class QTICamera2Client;
#endif
...
#ifdef USE_QTI_CAMERA2CLIENT
    sp<camera2::RawProcessor> mRawProcessor;
#endif
...
#ifdef USE_QTI_CAMERA2CLIENT
    sp<QTICamera2Client> mQTICamera2Client;
#endif
...
}

// 2. Camera2Client.cpp
template<typename TProviderPtr>
status_t Camera2Client::initializeImpl(TProviderPtr providerPtr)
{
...
#ifdef USE_QTI_CAMERA2CLIENT
    mQTICamera2Client = new QTICamera2Client(this);
#endif
...
#ifdef USE_QTI_CAMERA2CLIENT
    mRawProcessor = new RawProcessor(this, mCaptureSequencer);
    threadName = String8::format("C2-%d-RawProc", mCameraId);
    mRawProcessor->run(threadName.string());
#endif
...
}

QTICamera2Client 是高通对 API 1 中 Camera2Client 做的一层封装，添加了部分功能，主要是向上提供 raw 数据。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// 1. QTICamera2Client.h
class QTICamera2Client: public virtual RefBase{
private:
    wp<Camera2Client> mParentClient;
    status_t stopPreviewExtn();

public:
    QTICamera2Client(sp<Camera2Client> client);
    ~QTICamera2Client();
    ...
}

// 2. QTICamera2Client.cpp
QTICamera2Client::QTICamera2Client(sp<Camera2Client> client):
        mParentClient(client) {
}

device1/device3

device1/device3 可以理解为 Framework 层对应 HAL 层的 HAL 1/3 。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
device1
├── CameraHardwareInterface.cpp
└── CameraHardwareInterface.h
device3
├── Camera3BufferManager.cpp
├── Camera3BufferManager.h
├── Camera3Device.cpp
├── Camera3Device.h
├── Camera3DummyStream.cpp
├── Camera3DummyStream.h
├── Camera3InputStream.cpp
├── Camera3InputStream.h
├── Camera3IOStreamBase.cpp
├── Camera3IOStreamBase.h
├── Camera3OutputStream.cpp
├── Camera3OutputStream.h
├── Camera3OutputStreamInterface.h
├── Camera3SharedOutputStream.cpp
├── Camera3SharedOutputStream.h
├── Camera3StreamBufferFreedListener.h
├── Camera3StreamBufferListener.h
├── Camera3Stream.cpp
├── Camera3Stream.h
├── Camera3StreamInterface.h
├── Camera3StreamSplitter.cpp
├── Camera3StreamSplitter.h
├── StatusTracker.cpp
└── StatusTracker.h
  • API1/device1/HAL1 的连接过程 
1
2
3
4
5
// API1: CameraClient.h
sp<CameraHardwareInterface>     mHardware;
// device1: CameraHardwareInterface.h
sp<hardware::camera::device::V1_0::ICameraDevice> mHidlDevice;
// 这里的 ICameraDevice 即为 HAL1

API1 的客户端 CameraClient 对应的 device1: CameraHardwareInterface，而它直接包含了 HAL1 中 ICameraDevice 。

  • API1/3/device3/HAL3 的连接过程 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    // API1: Camera2Client.h
    class Camera2Client :
        public Camera2ClientBase<CameraService::Client>{...}

    // API2: CameraDeviceClient.h
    class CameraDeviceClient :
        public Camera2ClientBase<CameraDeviceClientBase>,
        public camera2::FrameProcessorBase::FilteredListener{...}

    // Camera2ClientBase.h
    sp<CameraDeviceBase>  mDevice;

    // Camera2ClientBase.cpp
    template <typename TClientBase>
    Camera2ClientBase<TClientBase>::Camera2ClientBase(
        const sp<CameraService>& cameraService,
        const sp<TCamCallbacks>& remoteCallback,
        const String16& clientPackageName,
        const String8& cameraId,
        int cameraFacing,
        int clientPid,
        uid_t clientUid,
        int servicePid):
        TClientBase(cameraService, remoteCallback, clientPackageName,
                cameraId, cameraFacing, clientPid, clientUid, servicePid),
        mSharedCameraCallbacks(remoteCallback),
        mDeviceVersion(cameraService->getDeviceVersion(
                TClientBase::mCameraIdStr)),
        mDeviceActive(false)
    {
        ...
        mInitialClientPid = clientPid;
        // 只要是 HAL3 ，则 device 都是对应的 Camera3Device
        mDevice = new Camera3Device(cameraId);
        ...
    }

从源码可以看出，不管是 API1/2 ，只要是 HAL 3 ，Camera2Client, CameraDeviceClient 两个客户端对应的都是 device3: Camera3Device 。

Camera3Device::HalInterface 内部类，用于和 HAL 层通信，实现了 HAL 层 ICameraDeviceSession.hal 部分代码： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// Camera3Device.h
class Camera3Device :
    ...
    class HalInterface : public camera3::Camera3StreamBufferFreedListener {
      public:
        ...
        // Calls into the HAL interface

        // Caller takes ownership of requestTemplate
        status_t constructDefaultRequestSettings(
                camera3_request_template_t templateId,
                /*out*/ camera_metadata_t **requestTemplate);
        status_t configureStreams(
            /*inout*/ camera3_stream_configuration *config);
        status_t processCaptureRequest(camera3_capture_request_t *request);
        status_t flush();
        status_t close();
        ...
    }
    ...
}

cameraserver 进程

cameraserver 进程的源码在 frameworks/av/camera/cameraserver 目录下，该目录只有三个文件： 

1
2
3
4
.
├── Android.mk              
├── cameraserver.rc         // rc 文件
└── main_cameraserver.cpp   // 主进程

cameraserver 进程在启动时，做了三件事：

  • 设置 Socket 通信时，对端关闭读取时进程不退出，返回错误信息（Socket 用在了哪？）
  • HIDL 通信初始化
  • Native Binder 初始化，CameraService 向 service_manager 注册服务 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// frameworks/av/camera/cameraserver/main_cameraserver.cpp
int main(int argc __unused, char** argv __unused)
{
    // 1. Socket 通信时，对端关闭读取时进程不退出，返回错误信息
    signal(SIGPIPE, SIG_IGN);

    // 2. HIDL 通信初始化
    // Set 3 threads for HIDL calls
    hardware::configureRpcThreadpool(3, /*willjoin*/ false);

    // 3. Native Binder 初始化，CameraService 是具体的服务
    sp<ProcessState> proc(ProcessState::self());
    sp<IServiceManager> sm = defaultServiceManager();
    ALOGI("ServiceManager: %p", sm.get());
    CameraService::instantiate();
    ProcessState::self()->startThreadPool();
    IPCThreadState::self()->joinThreadPool();
}
1
2
3
4
5
6
7
8
9
10
// init 进程启动名字为 cameraserver 的进程及对应路径
service cameraserver /system/bin/cameraserver
    // class 表示类别，同一类别的进程同时启动
    class main
    // 用户名
    user cameraserver
    // 分组
    group audio camera input drmrpc
    ioprio rt 4
    writepid /dev/cpuset/camera-daemon/tasks /dev/stune/top-app/tasks

CameraService 启动服务注册流程图：
0109-android-camera-4-framework-CameraService-addService.png

CameraService 服务

CameraService 服务的名称为：media.camera ，主要有两个功能：

  • 作为服务端
    实现 AIDL 对应功能，当 API1/2 客户端发出请求后，作为服务端响应并处理这些功能。
  • 作为客户端
    实现 HIDL 回调，用于响应 HAL 层发回的回调。并且通过 CameraProviderManager 和 HAL 层实现双向通信。

服务名称

CameraService 继承了 BinderService<CameraService> ，将 CameraService::instantiate(); 代码展开： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
// BinderService.h
template<typename SERVICE>
class BinderService
{
public:
    static status_t publish(bool allowIsolated = false) {
        sp<IServiceManager> sm(defaultServiceManager());
        return sm->addService(
                String16(SERVICE::getServiceName()),
                new SERVICE(), allowIsolated);
    }
    ...
    static void instantiate() { publish(); }
    ...
}

// IServiceManager.h
class IServiceManager : public IInterface
{
public:
    ...
    virtual status_t addService( const String16& name,
            const sp<IBinder>& service,
            bool allowIsolated = false) = 0;
    ...
}

// CameraService.h
class CameraService :
    public BinderService<CameraService>,
    public virtual ::android::hardware::BnCameraService,
    public virtual IBinder::DeathRecipient,
    public camera_module_callbacks_t,
    public virtual CameraProviderManager::StatusListener
{
    ...
    // Implementation of BinderService<T>
    static char const* getServiceName() { return "media.camera"; }
    ...
}

从继承关系及 CameraService.h 源码，getServiceName 设置了 CameraService 服务的名称为 media.camera 。

注册流程图

CameraService 注册流程，查看原图

0109-android-camera-4-framework-CameraService-init.jpg

源码分析

先来看 CameraService.h 头文件相关定义： 

1
2
3
4
5
6
7
8
9
10
11
// CameraService.h
class CameraService :
    public BinderService<CameraService>,
    public virtual ::android::hardware::BnCameraService,
    public virtual IBinder::DeathRecipient,
    public camera_module_callbacks_t,
    public virtual CameraProviderManager::StatusListener
{

    static char const* getServiceName() { return "media.camera"; }
}
  • BinderService
    继承了 BinderService ，用于注册服务。服务名称为 media.camera 。
  • camera_module_callbacks_t
    继承了 camera_module_callbacks_t ，它是在 HAL 中定义的，用于 HAL 向 Framework 发送通知。
  • StatusListener
    继承了 StatusListener ，它是在 CameraProviderManager.h 中定义的，用于 CameraProviderManager 向 CameraService 发送通知。

现在查看 CameraService 的构造方法，因为在注册服务时 BinderService 会对 CameraService 强指针引用，所以会调用对应函数 onFirstRef ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
// CameraService.cpp
CameraService::CameraService() :
        mEventLog(DEFAULT_EVENT_LOG_LENGTH),
        mNumberOfCameras(0), mNumberOfNormalCameras(0),
        mSoundRef(0), mInitialized(false) {
    // camera_module_callbacks_t 结构体的函数指针赋值
    this->camera_device_status_change = 
        android::camera_device_status_change;
    this->torch_mode_status_change = android::torch_mode_status_change;
    ...
}

void CameraService::onFirstRef()
{
    ...
    BatteryNotifier& notifier(BatteryNotifier::getInstance());
    notifier.noteResetCamera();
    notifier.noteResetFlashlight();

    status_t res = INVALID_OPERATION;

    // 实例化 CameraProviderManager ，并连接 Hardware
    res = enumerateProviders();
    if (res == OK) {
        mInitialized = true;
    }

    // CameraServiceProxy 服务是 Java 代码注册的
    // 但是 CameraService 启动时间很早，CameraServiceProxy 可能还并没有注册
    // 实际调试结果也是，这段代码实际不会调用 CameraServiceProxy 对应方法
    CameraService::pingCameraServiceProxy();
}

构造函数中非常简单，仅仅是将 camera_module_callbacks_t 结构体的函数指针赋值；在 onFirstRef 中，主要通过 enumerateProviders 来实例化对应的 CameraProviderManager 并连接 HAL ，最后去 ping 一次 CameraServiceProxy 代理服务，实际上是 ping 不通的，因为 CameraService.cpp 一定是比 CameraServiceProxy.java 启动的早。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
// CameraService.cpp
status_t CameraService::enumerateProviders() {
    ...
    // 实例化 CameraProviderManager
    if (nullptr == mCameraProviderManager.get()) {
        mCameraProviderManager = new CameraProviderManager();
        res = mCameraProviderManager->initialize(this);
        ...
    }

    mNumberOfCameras = mCameraProviderManager->getCameraCount();
    mNumberOfNormalCameras =
            mCameraProviderManager->getAPI1CompatibleCameraCount();
    mCameraProviderManager->setUpVendorTags();

    if (nullptr == mFlashlight.get()) {
        mFlashlight = new CameraFlashlight(mCameraProviderManager, this);
    }
    res = mFlashlight->findFlashUnits();
    ...
    for (auto& cameraId : mCameraProviderManager->getCameraDeviceIds()){
        ...
        onDeviceStatusChanged(id8, CameraDeviceStatus::PRESENT);
        ...
    }

    return OK;
}

如果 mCameraProviderManager 为空，则实例化并调用 initialize ；接着实例化 CameraFlashlight ；先看头文件 CameraProviderManager.h 中定义的几个重要数据结构和函数： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
// CameraProviderManager.h
class CameraProviderManager : 
    virtual public hidl::manager::V1_0::IServiceNotification {
public:
    ...
    // 定义纯虚函数
    struct ServiceInteractionProxy {
        virtual bool registerForNotifications(
                const std::string &serviceName,
                const sp<hidl::manager::V1_0::IServiceNotification>
                &notification) = 0;
        virtual sp<hardware::camera::provider::V2_4::ICameraProvider> 
                getService(const std::string &serviceName) = 0;
        virtual ~ServiceInteractionProxy() {}
    };

    // 调用 ICameraProvider 实现这些方法
    struct HardwareServiceInteractionProxy : 
            public ServiceInteractionProxy {
        virtual bool registerForNotifications(
                const std::string &serviceName,
                const sp<hidl::manager::V1_0::IServiceNotification>
                &notification) override {
            return hardware::camera::provider::V2_4::
                    ICameraProvider::registerForNotifications(
                    serviceName, notification);
        }
        virtual sp<hardware::camera::provider::V2_4::ICameraProvider>
                getService(const std::string &serviceName) override {
            return hardware::camera::provider::V2_4::
                    ICameraProvider::getService(serviceName);
        }
    };

    struct StatusListener : virtual public RefBase {
        ~StatusListener() {}
        virtual void onDeviceStatusChanged(const String8 &cameraId,
                hardware::camera::common::V1_0::CameraDeviceStatus 
                newStatus) = 0;
        virtual void onTorchStatusChanged(const String8 &cameraId,
                hardware::camera::common::V1_0::TorchModeStatus 
                newStatus) = 0;
        virtual void onNewProviderRegistered() = 0;
    };

    virtual hardware::Return<void> onRegistration(
            const hardware::hidl_string& fqName,
            const hardware::hidl_string& name,
            bool preexisting) override;

    status_t initialize(wp<StatusListener> listener,
            ServiceInteractionProxy *proxy 
            = &sHardwareServiceInteractionProxy);

private:
  static HardwareServiceInteractionProxy sHardwareServiceInteractionProxy;
  • ServiceInteractionProxy
    定义了几个纯虚函数，用于向 HAL 系统服务中注册 registerForNotifications 监听 ICameraProvider.hal 的消息；getService 返回 ICameraProvider 的实例。
  • HardwareServiceInteractionProxy
    ServiceInteractionProxy 的实现结构体，具体调用 ICameraProvider 对应的 registerForNotifications, getService ；也就是 CameraProviderManager 持有 ICameraProvider 的远程实例。
  • onRegistration
    registerForNotifications 的回调函数，注册成功后回调。
  • StatusListener
    状态监听接口，这些接口是在 CameraService 中实现的；用于 CameraProviderManager 回调 CameraService 。
  • sHardwareServiceInteractionProxy
    静态变量，是初始化 initialize 函数形参 ServiceInteractionProxy 的默认值。

从 CameraService 中调用 CameraProviderManager::initialize 时，传入的是 CameraService 的实例，仅仅一个参数，所以 ServiceInteractionProxy 使用的是默认的 sHardwareServiceInteractionProxy 实例。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// CameraProviderManager.cpp
// 实例化 HAL 代理
CameraProviderManager::HardwareServiceInteractionProxy
CameraProviderManager::sHardwareServiceInteractionProxy{};

status_t CameraProviderManager::initialize(
        wp<CameraProviderManager::StatusListener> listener,
        ServiceInteractionProxy* proxy) {
    ...
    mListener = listener;
    mServiceProxy = proxy;

    bool success = mServiceProxy->registerForNotifications(
        /* instance name, empty means no filter */ "",
        this);
    ...

    addProviderLocked(kLegacyProviderName, /*expected*/ false);
    return OK;
}

CameraProviderManager::initialize 中主要是初始化赋值 mListener, mServiceProxy ，并通过 sHardwareServiceInteractionProxy->registerForNotifications 向 HIDL 服务管理注册了自己，最后调用 addProviderLocked 。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
// CameraProviderManager.cpp
status_t CameraProviderManager::addProviderLocked(
        const std::string& newProvider, bool expected) {
    for (const auto& providerInfo : mProviders) {
        if (providerInfo->mProviderName == newProvider) {
            ALOGW(...);
            return ALREADY_EXISTS;
        }
    }

    // HIDL 通信，通过 ICameraProvider 和 HAL 层通信
    sp<provider::V2_4::ICameraProvider> interface;
    interface = mServiceProxy->getService(newProvider);

    if (interface == nullptr) {
        ...
    }

    sp<ProviderInfo> providerInfo =
            new ProviderInfo(newProvider, interface, this);
    status_t res = providerInfo->initialize();
    if (res != OK) {
        return res;
    }

    mProviders.push_back(providerInfo);

    return OK;
}

addProviderLocked 中有如下信息：

  • 通过代理获取 ICameraProvider 实例，用于和 HAL 通信
  • 新建 ProviderInfo 并初始化，保存 ICameraProvider 实例
  • mProviders 保存所有的 ProviderInfo （实测只有一个实例元素，名称为 legacy/0） 
1
2
3
4
5
6
7
8
9
10
11
12
// CameraProviderManager.h
struct ProviderInfo :
    virtual public 
        hardware::camera::provider::V2_4::ICameraProviderCallback,
    virtual public hardware::hidl_death_recipient
{
    const std::string mProviderName;
    const sp<hardware::camera::provider::V2_4::ICameraProvider> 
        mInterface;
    const metadata_vendor_id_t mProviderTagid;
    ...
}

ProviderInfo 继承了 ICameraProviderCallback, hidl_death_recipient ，它会处理来着 ICameraProvider 的回调。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
// CameraProviderManager.cpp
CameraProviderManager::ProviderInfo::ProviderInfo(
        const std::string &providerName,
        sp<provider::V2_4::ICameraProvider>& interface,
        CameraProviderManager *manager) :
        mProviderName(providerName),
        mInterface(interface),
        mProviderTagid(generateVendorTagId(providerName)),
        mUniqueDeviceCount(0),
        mManager(manager) {
    (void) mManager;
}

status_t CameraProviderManager::ProviderInfo::initialize() {
    status_t res = parseProviderName(mProviderName, &mType, &mId);
    ...
    // 设置回调
    hardware::Return<Status> status = mInterface->setCallback(this);
    ...
    // HIDL 连接
    hardware::Return<bool> linked = 
        mInterface->linkToDeath(this, /*cookie*/ mId);
    ...
    // Get initial list of camera devices, if any
    std::vector<std::string> devices;
    // 获取 CameraIdList ，实际是获取的一组设备名
    hardware::Return<void> ret = 
        mInterface->getCameraIdList([&status, &devices](
            Status idStatus,
            const hardware::hidl_vec<hardware::hidl_string>& 
                cameraDeviceNames) {
        status = idStatus;
        if (status == Status::OK) {
            for (size_t i = 0; i < cameraDeviceNames.size(); i++) {
                devices.push_back(cameraDeviceNames[i]);
            }
        } });
    ...
    for (auto& device : devices) {
        std::string id;
        // 添加从 HAL 返回的每个设备名
        status_t res = addDevice(device,
            hardware::camera::common::V1_0::CameraDeviceStatus::PRESENT,
            &id);
        ...
    }
    ...
    return OK;
}

ProviderInfo::initialize 初始化，主要是从 HAL 获取设备名后，添加具体的设备信息。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
// CameraProviderManager.h
struct DeviceInfo {
    const std::string mName;  // Full instance name
    const std::string mId;    // ID section of full name
    const hardware::hidl_version mVersion;
    const metadata_vendor_id_t mProviderTagid;
    ...
protected:
    bool mHasFlashUnit;
    template<class InterfaceT>
    static status_t setTorchMode(InterfaceT& interface, bool enabled);
};

// HALv1-specific camera fields, including the actual device interface
struct DeviceInfo1 : public DeviceInfo {
    typedef hardware::camera::device::V1_0::ICameraDevice InterfaceT;
    const sp<InterfaceT> mInterface;
    ...
private:
    CameraParameters2 mDefaultParameters;
};

// HALv3-specific camera fields, including the actual device interface
struct DeviceInfo3 : public DeviceInfo {
    typedef hardware::camera::device::V3_2::ICameraDevice InterfaceT;
    const sp<InterfaceT> mInterface;
    ...
private:
    CameraMetadata mCameraCharacteristics;
};

头文件中可以看出，DeviceInfo 有两个子类，分别对应 HAL 1 和 HAL 3 ，并将具体的 ICameraDevice 版本保存到 mInterface 中；所以设备添加时也会根据不同版本分别添加： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
// CameraProviderManager.cpp
status_t CameraProviderManager::ProviderInfo::addDevice(
        const std::string& name,
        CameraDeviceStatus initialStatus, 
        /*out*/ std::string* parsedId) {
    ...
    status_t res = parseDeviceName(name, &major, &minor, &type, &id);
    ...
    std::unique_ptr<DeviceInfo> deviceInfo;
    switch (major) {
        case 1:
            deviceInfo = initializeDeviceInfo<DeviceInfo1>(
                name, mProviderTagid, id, minor);
            break;
        case 3:
            deviceInfo = initializeDeviceInfo<DeviceInfo3>(
                name, mProviderTagid, id, minor);
            break;
        default:
            ALOGE(...);
            return BAD_VALUE;
    }
    ...
    return OK;
}

template<class DeviceInfoT>
std::unique_ptr<CameraProviderManager::ProviderInfo::DeviceInfo>
    CameraProviderManager::ProviderInfo::initializeDeviceInfo(
        const std::string &name, const metadata_vendor_id_t tagId,
        const std::string &id, uint16_t minorVersion) const {
    Status status;

    auto cameraInterface =
            getDeviceInterface<typename DeviceInfoT::InterfaceT>(name);
    if (cameraInterface == nullptr) return nullptr;

    CameraResourceCost resourceCost;
    cameraInterface->getResourceCost([&status, &resourceCost](
        Status s, CameraResourceCost cost) {
                status = s;
                resourceCost = cost;
            });
    ...
    return std::unique_ptr<DeviceInfo>(
        new DeviceInfoT(name, tagId, id, minorVersion, resourceCost,
                cameraInterface));
}

根据传入的 deviceName 解析版本号、类型、设备 Id （前后摄），并根据 major 版本号（表示 HAL 1 或者 HAL 3） 分别初始化对应的 DeviceInfo ；在 initializeDeviceInfo 中通过 getDeviceInterface 获取对应的 ICameraDevice 版本，在对应版本 DeviceInfo 实例化时保存；也就是将 DeviceInfo 和 HAL 层的 ICameraDevice 绑定。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
// CameraProviderManager.cpp
CameraProviderManager::ProviderInfo::DeviceInfo3::DeviceInfo3(
        const std::string& name,
        const metadata_vendor_id_t tagId, const std::string &id,
        uint16_t minorVersion,
        const CameraResourceCost& resourceCost,
        sp<InterfaceT> interface) :
        DeviceInfo(name, tagId, id, 
            hardware::hidl_version{3, minorVersion}, resourceCost),
        mInterface(interface) {
    // Get camera characteristics and initialize flash unit availability
    Status status;
    hardware::Return<void> ret;
    // 获取 Camera 设备配置信息
    ret = mInterface->getCameraCharacteristics([&status, this](Status s,
                device::V3_2::CameraMetadata metadata) {
            status = s;
            if (s == Status::OK) {
                camera_metadata_t *buffer =
                    reinterpret_cast<camera_metadata_t*>(metadata.data());
                size_t expectedSize = metadata.size();
                int res = validate_camera_metadata_structure(buffer, 
                    &expectedSize);
                if (res==OK||res==CAMERA_METADATA_VALIDATION_SHIFTED) {
                    set_camera_metadata_vendor_id(buffer, mProviderTagid);
                    mCameraCharacteristics = buffer;
                } else {
                    ALOGE(...);
                    status = Status::INTERNAL_ERROR;
                }
            }
        });
    ...
    camera_metadata_entry flashAvailable =
            mCameraCharacteristics.find(ANDROID_FLASH_INFO_AVAILABLE);
    if (flashAvailable.count == 1 &&
            flashAvailable.data.u8[0] == ANDROID_FLASH_INFO_AVAILABLE_TRUE) {
        mHasFlashUnit = true;
    } else {
        mHasFlashUnit = false;
    }
}

这里分析的是 DeviceInfo3 的构造函数，它会向 HAL 层请求当前设备的配置信息，并保存 mCameraCharacteristics ，后续查看属性时都会通过这个变量查询。
CameraService::enumerateProviders 中，首先新建并初始化 CameraProviderManager ，其持有和 HAL 通信的实例；接着新建并初始化 CameraFlashlight ，用于控制闪光灯。先看头文件： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
// CameraFlashlight.h
class FlashControlBase : public virtual VirtualLightRefBase {
public:
    ...
    virtual status_t hasFlashUnit(const String8& cameraId,
                bool *hasFlash) = 0;
    virtual status_t setTorchMode(const String8& cameraId,
                bool enabled) = 0;
};

// HAL 3 闪光灯控制
class ProviderFlashControl : public FlashControlBase {
public:
    ...
    // FlashControlBase
    status_t hasFlashUnit(const String8& cameraId, bool *hasFlash);
    status_t setTorchMode(const String8& cameraId, bool enabled);
private:
    sp<CameraProviderManager> mProviderManager;
...
};

// HAL 1 闪光灯控制，通过 CameraHardwareInterface 向下调用
class CameraHardwareInterfaceFlashControl : public FlashControlBase {
public:
    ...
    // FlashControlBase
    status_t setTorchMode(const String8& cameraId, bool enabled);
    status_t hasFlashUnit(const String8& cameraId, bool *hasFlash);
private:
    sp<CameraProviderManager> mProviderManager;
    const camera_module_callbacks_t *mCallbacks;
    sp<CameraHardwareInterface> mDevice;
    String8 mCameraId;
    CameraParameters mParameters;
    ...
}

class CameraFlashlight : public virtual VirtualLightRefBase {
public:
    ...
    bool hasFlashUnit(const String8& cameraId);
    status_t setTorchMode(const String8& cameraId, bool enabled);
private:
    sp<FlashControlBase> mFlashControl;
    sp<CameraProviderManager> mProviderManager;
    const camera_module_callbacks_t *mCallbacks;

头文件定义的几个信息：

  • CameraHardwareInterfaceFlashControl
    HAL 1 闪光灯控制类，通过 CameraHardwareInterface 向下调用。
  • ProviderFlashControl
    HAL 3 闪光灯控制类。
  • FlashControlBase
    基类。
  • CameraProviderManager
    主要用于 ProviderFlashControl 向下发送信息。
  • camera_module_callbacks_t
    HAL 层的回调。 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
// CameraFlashlight.cpp
CameraFlashlight::CameraFlashlight(
        sp<CameraProviderManager> providerManager,
        camera_module_callbacks_t* callbacks) :
        mProviderManager(providerManager),
        mCallbacks(callbacks),
        mFlashlightMapInitialized(false) {
}

status_t CameraFlashlight::findFlashUnits() {
    ...
    mFlashControl.clear();
    for (auto &id : cameraIds) {
        ssize_t index = mHasFlashlightMap.indexOfKey(id);
        if (0 <= index) {
            continue;
        }

        bool hasFlash = false;
        res = createFlashlightControl(id);
        ...
    }
    ...
    return OK;
}

status_t CameraFlashlight::createFlashlightControl(
        const String8& cameraId) {
    ...
    if (mProviderManager->supportSetTorchMode(cameraId.string())) {
        mFlashControl = new ProviderFlashControl(mProviderManager);
    } else {
        // Only HAL1 devices do not support setTorchMode
        mFlashControl =
            new CameraHardwareInterfaceFlashControl(mProviderManager,
            *mCallbacks);
    }
    return OK;
}

CameraFlashlight 的构造函数仅仅初始化了几个本地变量，CameraService 中调用 CameraFlashlight::findFlashUnits 时，会根据 HAL 1/3 分别来创建对应的闪光灯控制类。至此整个 CameraService 注册流程结束。

小结

CameraService 初始化和注册流程中，实例化了两个对象：

  • CameraProviderManager mCameraProviderManager 对象
  • Flashlight mFlashlight 对象

CameraProviderManager 初始化完后：

  • mProviders 保存了 ProviderInfo 对象；并关联了 ICameraProvider ，用于和 HAL 通信
  • ProviderInfo 中 mDevices 保存了所有的 DeviceInfo1, DeviceInfo3 设备信息，并关联 ICameraDevice 实例，用于直接通信
  • DeviceInfo1 中保存了 CameraParameters2 mDefaultParameters 参数信息
  • DeviceInfo3 中保存了 CameraMetadata mCameraCharacteristics 参数信息

CameraFlashlight 新建和初始化后：

  • 如果是 HAL 1 会实例化控制类 CameraHardwareInterfaceFlashControl
  • 如果是 HAL 3 会实例化控制类 ProviderFlashControl

Camera Open 流程

API

Camera API 2 开启摄像头设备时，通过 CameraManager.openCamera 来打开： 

1
2
3
4
5
6
7
8
9
// CameraManager.java
@RequiresPermission(android.Manifest.permission.CAMERA)
public void openCamera(@NonNull String cameraId,
        @NonNull final CameraDevice.StateCallback callback,
        @Nullable Handler handler)
        throws CameraAccessException {

    openCameraForUid(cameraId, callback, handler, USE_CALLING_UID);
}
  • String cameraId
    表示前后摄的 ID ，通常 0 表示后摄。
  • CameraDevice.StateCallback callback
    打开设备时，状态回调接口。
  • Handler handler
    表示回调接口在哪个线程执行。

示例

打开一个设备，在回调中保存 CameraDevice ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
CameraDevice.StateCallback mCameraDeviceStateCallback 
        = new CameraDevice.StateCallback() {
    @Override
    public void onOpened(@NonNull CameraDevice camera) {
        mCameraDevice = camera;
        //createCameraCaptureSession();
    }

    @Override
    public void onDisconnected(@NonNull CameraDevice camera) {
        camera.close();
        mCameraDevice = null;
    }

    @Override
    public void onError(@NonNull CameraDevice camera, int error) {
        camera.close();
        mCameraDevice = null;
    }
};

try {
    mCameraManager.openCamera(mCameraId, mCameraDeviceStateCallback,
            mBackHandler);
} catch (CameraAccessException e) {
    e.printStackTrace();
}

CameraDevice.StateCallback 接口

在打开设备时，会传入 StateCallback 回调接口，它有四个方法，都是在 CameraDeviceImpl 中回调的：

  • onOpened
    在 CameraManager.openCameraDeviceUserAsync 方法中，CameraDeviceImpl.setRemoteDevice(cameraUser); 会触发 StateCallback.onOpened 回调。
  • onClosed
    CameraDevice.close 是在 CameraDeviceImpl.close 中实现的，同时会触发 StateCallback.onClosed 回调。
  • onDisconnected
    CameraDeviceImpl.setRemoteDevice(cameraUser); 中如果远程连接断开，或者 ICameraDeviceCallbacks.onDeviceError 返回了 ERROR_CAMERA_DISCONNECTED 错误码，都会触发 StateCallback.onDisconnected 回调。
  • onError
    在 Binder 通信中绑定失败 binderDied ，setRemoteFailure 以及 ICameraDeviceCallbacks.onDeviceError 返回了 ERROR_CAMERA_DEVICE/ERROR_CAMERA_SERVICE 错误码，都会触发 StateCallback.onError 回调。

在设备打开时，会通过 StateCallback 回调返回打开状态，从代码可以看出，只要 ICameraService.connectDevice 成功后，直接调用 CameraDeviceImpl.setRemoteDevice(cameraUser); 来触发 StateCallback.onOpened ，表示设备打开成功。
StateCallback 是 Java 接口，它的 onDisconnected, onError 两个回调方法，需要真实的与物理设备交互；所以需要通过 ICameraDeviceCallbacks.aidl 从 Framework Service 中获取真实的信息回调。

流程图

Camera API 2 开启相机设备流程图：

0109-android-camera-4-framework-Camera2-open.png

0109-android-camera-4-framework-Camera2-open-CameraService_connectDevice.jpg

源码分析

通过 CameraManager.openCamera 打开设备，我们重点分析如下代码，代码执行路径为 ：
openCamera -> openCameraForUid -> openCameraDeviceUserAsync 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
// CameraManager.java
@RequiresPermission(android.Manifest.permission.CAMERA)
public void openCamera(@NonNull String cameraId,
        @NonNull final CameraDevice.StateCallback callback, 
        @Nullable Handler handler)
        throws CameraAccessException {

    openCameraForUid(cameraId, callback, handler, USE_CALLING_UID);
}

private CameraDevice openCameraDeviceUserAsync(String cameraId,
        CameraDevice.StateCallback callback, 
        Handler handler, final int uid)
        throws CameraAccessException {
    CameraCharacteristics characteristics = 
        getCameraCharacteristics(cameraId);
    CameraDevice device = null;

    synchronized (mLock) {
        ICameraDeviceUser cameraUser = null;
        // 新建 CameraDeviceImpl 实例
        android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
                new android.hardware.camera2.impl.CameraDeviceImpl(
                    cameraId,
                    callback,
                    handler,
                    characteristics,
                    mContext.getApplicationInfo().targetSdkVersion);
        // 获取 ICameraDeviceCallbacks 回调
        ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();

        try {
            if (supportsCamera2ApiLocked(cameraId)) {
                // Use cameraservice's cameradeviceclient 
                // implementation for HAL3.2+ devices
                ICameraService cameraService = 
                    CameraManagerGlobal.get().getCameraService();
                if (cameraService == null) {
                    throw new ServiceSpecificException(...);
                }
                // 连接设备，并获取 ICameraDeviceUser
                cameraUser = cameraService.connectDevice(callbacks,
                        cameraId, mContext.getOpPackageName(), uid);
            } else {
                // Use legacy camera implementation for HAL1 devices
                int id;
                try {
                    id = Integer.parseInt(cameraId);
                } catch (NumberFormatException e) {
                    throw new IllegalArgumentException(...);
                }

                Log.i(TAG, "Using legacy camera HAL.");
                cameraUser = 
                    CameraDeviceUserShim.connectBinderShim(callbacks, id);
            }
        } catch (ServiceSpecificException e) {
            ...
        } catch (RemoteException e) {
            ...
        }
        // 关联 CameraDeviceImpl 和 ICameraDeviceUser ，方便直接通信
        deviceImpl.setRemoteDevice(cameraUser);
        device = deviceImpl;
    }

    return device;
}

// CameraDeviceImpl.java
public void setRemoteDevice(ICameraDeviceUser remoteDevice) 
    throws CameraAccessException {
    synchronized(mInterfaceLock) {
        ...
        // 新建包装类，包装接口并处理对应访问异常
        mRemoteDevice = new ICameraDeviceUserWrapper(remoteDevice);
        ...
    }
}

从上面展示的 API 部分代码中可以看出：

  • 支持 API 2
    如果系统开启了 HAL 3 ，则支持 API 2 ；此时通过 ICameraService 访问服务。
  • 不支持 API 2
    如果系统仅支持 HAL 1 ，则 API 2 需要通过 CameraDeviceUserShim 转换为对应的 API 1 + HAL 1 来实现对应功能。CameraDeviceUserShim 是 ICameraDeviceUser 的实现类；整个 frameworks/base/core/java/android/hardware/camera2/legacy 目录下的代码都是为了实现这个转换功能。

整个打开设备的动作有如下功能：

  • 新建了 CameraDeviceImpl 实例，它是 CameraDevice 的实现类
  • CameraManager 通过 CameraService.connectDevice 连接设备，获取到 ICameraDeviceUser, ICameraDeviceCallbacks 对象，它们用于后续 CameraDeviceImpl.java 和 CameraDeviceClient.cpp 绑定通信
  • 新建 ICameraDeviceUserWrapper 实例，它是对 ICameraDeviceUser 的包装类，捕获并处理远程访问异常等

这里需要重点分析 connectDevice ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
// CameraService.cpp
Status CameraService::connectDevice(
        const sp<hardware::camera2::ICameraDeviceCallbacks>& cameraCb,
        const String16& cameraId,
        const String16& clientPackageName,
        int clientUid,
        /*out*/
        sp<hardware::camera2::ICameraDeviceUser>* device) {

    ATRACE_CALL();
    Status ret = Status::ok();
    String8 id = String8(cameraId);
    sp<CameraDeviceClient> client = nullptr;
    ret = connectHelper<hardware::camera2::ICameraDeviceCallbacks,
            CameraDeviceClient>(cameraCb, id,
            CAMERA_HAL_API_VERSION_UNSPECIFIED, clientPackageName,
            clientUid, USE_CALLING_PID, API_2,
            /*legacyMode*/ false, /*shimUpdateOnly*/ false,
            /*out*/client);

    ...
    *device = client;
    return ret;
}

template<class CALLBACK, class CLIENT>
Status CameraService::connectHelper(const sp<CALLBACK>& cameraCb,
        const String8& cameraId, int halVersion,
        const String16& clientPackageName, int clientUid, int clientPid,
        apiLevel effectiveApiLevel, bool legacyMode, bool shimUpdateOnly,
        /*out*/sp<CLIENT>& device) {
    ...
    {
        sp<BasicClient> tmp = nullptr;
        if(!(ret = makeClient(this, cameraCb, clientPackageName, 
                cameraId, facing, clientPid, clientUid, getpid(),
                legacyMode, halVersion, deviceVersion, effectiveApiLevel,
                /*out*/&tmp)).isOk()) {
            return ret;
        }
        ...
        err = client->initialize(mCameraProviderManager);
        ...
        if (shimUpdateOnly) {
            mServiceLock.unlock();
            client->disconnect();
            mServiceLock.lock();
        } else {
            // Otherwise, add client to active clients list
            finishConnectLocked(client, partial);
        }
    } // lock is destroyed, allow further connect calls

    device = client;
    return ret;
}

CameraService::connectDevice 函数调用了模板函数 connectHelper ，而该模板主要的两个功能就是：makeClient 新建客户端，initialize 初始化客户端。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
Status CameraService::makeClient(const sp<CameraService>& cameraService,
        const sp<IInterface>& cameraCb, const String16& packageName,
        const String8& cameraId, int facing, int clientPid, 
        uid_t clientUid, int servicePid, bool legacyMode,
        int halVersion, int deviceVersion, apiLevel effectiveApiLevel,
        /*out*/sp<BasicClient>* client) {

    if (halVersion < 0 || halVersion == deviceVersion) {
        switch(deviceVersion) {
          case CAMERA_DEVICE_API_VERSION_1_0:
            if (effectiveApiLevel == API_1) {  // Camera1 API route
                sp<ICameraClient> tmp = 
                    static_cast<ICameraClient*>(cameraCb.get());
                *client = new CameraClient(cameraService, tmp, 
                    packageName, cameraIdToInt(cameraId), facing, 
                    clientPid, clientUid, getpid(), legacyMode);
            } else { // Camera2 API route
                ALOGW("Camera using old HAL version: %d", deviceVersion);
                return STATUS_ERROR_FMT(...);
            }
            break;
          case CAMERA_DEVICE_API_VERSION_3_0:
          case CAMERA_DEVICE_API_VERSION_3_1:
          case CAMERA_DEVICE_API_VERSION_3_2:
          case CAMERA_DEVICE_API_VERSION_3_3:
          case CAMERA_DEVICE_API_VERSION_3_4:
            if (effectiveApiLevel == API_1) { // Camera1 API route
                sp<ICameraClient> tmp = 
                    static_cast<ICameraClient*>(cameraCb.get());
                *client = new Camera2Client(cameraService, tmp, 
                    packageName, cameraIdToInt(cameraId), facing,
                    clientPid, clientUid, servicePid, legacyMode);
            } else { // Camera2 API route
                sp<hardware::camera2::ICameraDeviceCallbacks> tmp =
                        static_cast<hardware::camera2::
                            ICameraDeviceCallbacks*>(cameraCb.get());
                *client = new CameraDeviceClient(cameraService, tmp, 
                    packageName, cameraId, facing, clientPid,
                    clientUid, servicePid);
            }
            break;
          default:
            // Should not be reachable
            ALOGE("Unknown camera device HAL version:%d", deviceVersion);
            return STATUS_ERROR_FMT(...);
        }
    } else {
        if (deviceVersion > CAMERA_DEVICE_API_VERSION_1_0 &&
            halVersion == CAMERA_DEVICE_API_VERSION_1_0) {
            sp<ICameraClient> tmp = 
                static_cast<ICameraClient*>(cameraCb.get());
            *client = new CameraClient(cameraService, tmp, packageName,
                cameraIdToInt(cameraId), facing, clientPid, 
                clientUid, servicePid, legacyMode);
        } else {
            ALOGE("Invalid camera HAL version ..");
            return STATUS_ERROR_FMT(...;
        }
    }
    return Status::ok();
}

makeClient 主要是根据 device, HAL 版本和调用 API 的版本来创建对应的客户端：

  • HAL 1 + API 1 ：新建 CameraClient
  • HAL 1 + API 2 ：不支持
  • HAL 3 + API 1 ：新建 Camera2Client
  • HAL 3 + API 2 ：新建 CameraDeviceClient

这里的三个变量 effectiveApiLevel, legacyMode=0, halVersion ，主要是有三个连接函数决定： connect, connectLegacy, connectDevice ，其中 connectLegacy 可以指定 HAL 版本（来决定到底使用哪个 client）：

  • 使用系统自带相机
    effectiveApiLevel=1, legacyMode=1, halVersion=256(HAL 1) ，系统自带应用使用的是 connectLegacy 。
  • 使用标准 API2 接口
    effectiveApiLevel=2, legacyMode=0, halVersion=-1 ，其中 -1 表示 CAMERA_HAL_API_VERSION_UNSPECIFIED 。

所谓的 HAL 版本，实际指的就是 Device 的版本：其中 HAL 1 对应 CAMERA_DEVICE_API_VERSION_1_0 ；HAL 3 对应的是 CAMERA_DEVICE_API_VERSION_3_0 及以上版本。而 HAL 2 和 CAMERA_DEVICE_API_VERSION_2_0 已经废弃。
因为手机平台使用 HAL 3 时，为了满足部分应用中使用了 API 1 的接口，常常需要兼容 HAL 1，所以支持 HAL 3 即意味着同时会支持 HAL 1 。

这里流程跟踪的是新建 CameraDeviceClient ，先看头文件： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
// CameraService.h
class BasicClient : public virtual RefBase {
public:
    virtual status_t initialize(sp<CameraProviderManager> manager) = 0;
    ...
private:
    ...
    class OpsCallback : public BnAppOpsCallback {
    public:
        explicit OpsCallback(wp<BasicClient> client);
        virtual void opChanged(int32_t op, const String16& packageName);

    private:
        wp<BasicClient> mClient;

    }; // class OpsCallback

    sp<OpsCallback> mOpsCallback;
    ...
}

// Camera3Device.h
class Camera3Device :
    public CameraDeviceBase,
    virtual public hardware::camera::device::V3_2::ICameraDeviceCallback,
    private camera3_callback_ops 
{...}

// Camera2ClientBase.h
template <typename TClientBase>
class Camera2ClientBase :
        public TClientBase,
        public CameraDeviceBase::NotificationListener
{
public:
    typedef typename TClientBase::TCamCallbacks TCamCallbacks;
    ...
protected:
    // 实例为 Camera3Device
    sp<CameraDeviceBase>  mDevice;
    ...
private:
    ...
    template<typename TProviderPtr>
    status_t              initializeImpl(TProviderPtr providerPtr);
};

// CameraDeviceClient.h
struct CameraDeviceClientBase :
         public CameraService::BasicClient,
         public hardware::camera2::BnCameraDeviceUser
{
    typedef hardware::camera2::ICameraDeviceCallbacks TCamCallbacks;
    ...

protected:
    CameraDeviceClientBase(const sp<CameraService>& cameraService,
        const sp<hardware::camera2::ICameraDeviceCallbacks>& 
            remoteCallback,
        const String16& clientPackageName,
        const String8& cameraId,
        int cameraFacing,
        int clientPid,
        uid_t clientUid,
        int servicePid);

    sp<hardware::camera2::ICameraDeviceCallbacks> mRemoteCallback;
};

class CameraDeviceClient :
        public Camera2ClientBase<CameraDeviceClientBase>,
        public camera2::FrameProcessorBase::FilteredListener
{...}

从类图结构来看：BasicClient 是三个客户端 CameraClient, Camera2Client, CameraDeviceClient 的基类；而 Camera2ClientBase 中的变量 CameraDeviceBase 实际的子类是 Camera3Device 。来看构造函数的流程： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
// CameraDeviceClient.cpp
CameraDeviceClient::CameraDeviceClient(
    const sp<CameraService>& cameraService,
    const sp<hardware::camera2::ICameraDeviceCallbacks>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraId,
    int cameraFacing,
    int clientPid,
    uid_t clientUid,
    int servicePid) :
    Camera2ClientBase(cameraService, remoteCallback, clientPackageName,
            cameraId, cameraFacing, clientPid, clientUid, servicePid),
    mInputStream(),
    mStreamingRequestId(REQUEST_ID_NONE),
    mRequestIdCounter(0),
    mPrivilegedClient(false) {
    ...
}

// Camera2ClientBase.cpp
template <typename TClientBase>
Camera2ClientBase<TClientBase>::Camera2ClientBase(
    const sp<CameraService>& cameraService,
    const sp<TCamCallbacks>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraId,
    int cameraFacing,
    int clientPid,
    uid_t clientUid,
    int servicePid):
    TClientBase(cameraService, remoteCallback, clientPackageName,
            cameraId, cameraFacing, clientPid, clientUid, servicePid),
    mSharedCameraCallbacks(remoteCallback),
    mDeviceVersion(cameraService->getDeviceVersion(
        TClientBase::mCameraIdStr)),
    mDeviceActive(false)
{
    ...
    // 实例化 Camera3Device
    mDevice = new Camera3Device(cameraId);
}

// CameraDeviceClient.cpp
CameraDeviceClientBase::CameraDeviceClientBase(
    const sp<CameraService>& cameraService,
    const sp<hardware::camera2::ICameraDeviceCallbacks>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraId,
    int cameraFacing,
    int clientPid,
    uid_t clientUid,
    int servicePid) :
    BasicClient(cameraService,
            IInterface::asBinder(remoteCallback),
            clientPackageName,
            cameraId,
            cameraFacing,
            clientPid,
            clientUid,
            servicePid),
    mRemoteCallback(remoteCallback) {
}

// CameraService.cpp
CameraService::BasicClient::BasicClient(
    const sp<CameraService>& cameraService,
    const sp<IBinder>& remoteCallback,
    const String16& clientPackageName,
    const String8& cameraIdStr, int cameraFacing,
    int clientPid, uid_t clientUid,
    int servicePid):
    mCameraIdStr(cameraIdStr), mCameraFacing(cameraFacing),
    mClientPackageName(clientPackageName), 
    mClientPid(clientPid), mClientUid(clientUid),
    mServicePid(servicePid),
    mDisconnected(false),
    mRemoteBinder(remoteCallback)
{
    ...
}

根据类继承关系，一条链路实例化各个子类，最终会新建 Camera3Device 实例。makeClient 新建完客户端后，调用客户端的初始化： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// CameraDeviceClient.cpp
template<typename TProviderPtr>
status_t CameraDeviceClient::initializeImpl(TProviderPtr providerPtr) {
    ...
    res = Camera2ClientBase::initialize(providerPtr);
    ...
    String8 threadName;
    mFrameProcessor = new FrameProcessorBase(mDevice);
    threadName = String8::format("CDU-%s-FrameProc", 
        mCameraIdStr.string());
    mFrameProcessor->run(threadName.string());

    mFrameProcessor->registerListener(FRAME_PROCESSOR_LISTENER_MIN_ID,
                                      FRAME_PROCESSOR_LISTENER_MAX_ID,
                                      /*listener*/this,
                                      /*sendPartials*/true);

    return OK;
}

CameraDeviceClient::initializeImpl 是一个模板函数，主要有两个功能：调用 Camera2ClientBase 及其父类初始化；新建 FrameProcessorBase 实例，它主要功能是在发出预览、拍照、录像请求后，HAL 层向 Framework 层返回结果的回调类，后面讲预览流程时会详细分析。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// Camera2ClientBase.cpp
template <typename TClientBase>
status_t Camera2ClientBase<TClientBase>::initialize(
    sp<CameraProviderManager> manager) {
    return initializeImpl(manager);
}

template <typename TClientBase>
template <typename TProviderPtr>
status_t Camera2ClientBase<TClientBase>::initializeImpl(
    TProviderPtr providerPtr) {
    ...
    res = mDevice->initialize(providerPtr);
    ...
    return OK;
}

Camera2ClientBase::initialize 也是一个模板函数，最终会调用 Camera3Device 的初始化： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
// Camera3Device.cpp
status_t Camera3Device::initialize(sp<CameraProviderManager> manager) {
    ...
    sp<ICameraDeviceSession> session;
    // 打开会话，获取 ICameraDeviceSession
    status_t res = manager->openSession(mId.string(), this,
            /*out*/ &session);
    ...
    // 获取当前设备的配置信息，并保存到 CameraMetadata mDeviceInfo
    res = manager->getCameraCharacteristics(mId.string(), &mDeviceInfo);
    ...
    // 通过 ICameraDeviceSession 获取请求队列
    std::shared_ptr<RequestMetadataQueue> queue;
    auto requestQueueRet = session->getCaptureRequestMetadataQueue(
        [&queue](const auto& descriptor) {
            queue = std::make_shared<RequestMetadataQueue>(descriptor);
            ...
        });
    ...
    // 通过 ICameraDeviceSession 获取结果队列
    std::unique_ptr<ResultMetadataQueue>& resQueue=mResultMetadataQueue;
    auto resultQueueRet = session->getCaptureResultMetadataQueue(
        [&resQueue](const auto& descriptor) {
            resQueue=std::make_unique<ResultMetadataQueue>(descriptor);
            ...
        });
    ...
    // 新建 HalInterface 实例，并绑定 ICameraDeviceSession 以及请求队列
    mInterface = new HalInterface(session, queue);
    ...
    return initializeCommonLocked();
}

Camera3Device::initialize 初始化中，重点实现的功能为打开物理设备，并获取 ICameraDeviceSession 用于后续直接和 HAL 通信，并通过它从 HAL 获取请求队列和结果队列；最后新建 HalInterface 实例，并将 ICameraDeviceSession 保存并绑定。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
// CameraProviderManager.cpp
status_t CameraProviderManager::openSession(const std::string &id,
    const sp<hardware::camera::device::V3_2::ICameraDeviceCallback>&
    callback, /*out*/
    sp<hardware::camera::device::V3_2::ICameraDeviceSession> *session){

    std::lock_guard<std::mutex> lock(mInterfaceMutex);

    auto deviceInfo = findDeviceInfoLocked(id,
            /*minVersion*/ {3,0}, /*maxVersion*/ {4,0});
    if (deviceInfo == nullptr) return NAME_NOT_FOUND;

   auto *deviceInfo3=static_cast<ProviderInfo::DeviceInfo3*>(deviceInfo);

    Status status;
    hardware::Return<void> ret;
    // 向 HAL 打开设备，并返回 ICameraDeviceSession
    ret = deviceInfo3->mInterface->open(callback, 
        [&status, &session] (Status s, 
        const sp<device::V3_2::ICameraDeviceSession>& cameraSession) {
            status = s;
            if (status == Status::OK) {
                *session = cameraSession;
            }
        });
    ...
}

CameraProviderManager::openSession 打开设备时，会向 HAL 打开设备，将 ICameraDeviceCallback 传入 HAL 并获取 ICameraDeviceSession 实例。接着看 Camera3Device::initializeCommonLocked ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Camera3Device.cpp
status_t Camera3Device::initializeCommonLocked() {

    /** Start up status tracker thread */
    mStatusTracker = new StatusTracker(this);
    status_t res = mStatusTracker->run(
        String8::format("C3Dev-%s-Status", mId.string()).string());
    ...

    /** Register in-flight map to the status tracker */
    mInFlightStatusId = mStatusTracker->addComponent();

    /** Create buffer manager */
    mBufferManager = new Camera3BufferManager();

    mTagMonitor.initialize(mVendorTagId);

    /** Start up request queue thread */
    mRequestThread = new RequestThread(this, mStatusTracker, mInterface);
    res = mRequestThread->run(
        String8::format("C3Dev-%s-ReqQueue", mId.string()).string());
    ...
    mPreparerThread = new PreparerThread();
    ...
    return OK;
}

在 initializeCommonLocked 中新建了很多实例：

  • StatusTracker ：状态跟踪线程
  • Camera3BufferManager ：输出流的图形缓冲区管理，比如 Camera3OutputStream 的管理
  • TagMonitor ：相机元数据 metadata 的监视器，比如 3A 信息等
  • RequestThread ：请求线程，比如拍照、录像、预览的数据请求
  • PreparerThread ：监测数据已经准备好流的线程

小结

以上流程图都是基于 API 2 + HAL 3，当 Camera Open 流程结束后：

  • 客户端调用 API 时，得到了 CameraDevice 的实例，并将 ICameraDeviceUser 和 CameraDeviceImpl 绑定
  • 根据 HAL 1/3 生成了对应的 Device 客户端，当前生成的是 CameraDeviceClient 实例
  • Camera3Device 在初始化时，调用 CameraProviderManager.openSession ，它会通过 HIDL 通知 HAL 层打开摄像头物理设备；打开成功会 Camera3Device::HalInterface 和 ICameraDeviceSession 实例绑定
  • 新建 RequestThread 对象，后台运行线程，用于监听 API 发起的请求 CaptureRequest ：预览、拍照、录像等
  • 新建 FrameProcessorBase 对象，后台运行线程，用于监听 HAL 返回的请求结果 CaptureResult

打开设备时，实际上 Framework, HAL 已经创建好会话 ICameraDeviceSession ；而下面分析的 API 创建会话流程，实际是根据不同需求（预览、拍照、录像）来创建和配置输出流。

创建会话流程

API

在打开设备后，获取到了 CameraDevice 的实例，通过它来创建会话 Session ： 

1
2
3
4
5
// CameraDevice.java
public abstract void createCaptureSession(@NonNull List<Surface> outputs,
        @NonNull CameraCaptureSession.StateCallback callback, 
        @Nullable Handler handler)
        throws CameraAccessException;
  • List<Surface> outputs
    表示有多少个输出流，通常为预览流和拍照/录像，两个输出流。
  • CameraCaptureSession.StateCallback callback
    创建会话状态回调。
  • Handler handler
    回调方法使用哪个线程响应，如果为 null 表示当前线程。

API 创建会话过程源码分析： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
// CameraDeviceImpl.java
@Override
public void createCaptureSession(List<Surface> outputs,
        CameraCaptureSession.StateCallback callback, Handler handler)
        throws CameraAccessException {
    // 将 Surface 转换为 OutputConfiguration
    List<OutputConfiguration> outConfigurations = 
        new ArrayList<>(outputs.size());
    for (Surface surface : outputs) {
        outConfigurations.add(new OutputConfiguration(surface));
    }
    createCaptureSessionInternal(null, outConfigurations, callback, 
            handler, ICameraDeviceUser.NORMAL_MODE);
}

private void createCaptureSessionInternal(InputConfiguration inputConfig,
        List<OutputConfiguration> outputConfigurations,
        CameraCaptureSession.StateCallback callback, Handler handler,
        int operatingMode) throws CameraAccessException {
    ...
    // 创建会话时，输入 Surface 为空
    Surface input = null;
    try {
        // configure streams and then block until IDLE
        // 向 Framework, HAL 发送信息，配置设备
        configureSuccess = configureStreamsChecked(inputConfig, 
            outputConfigurations, operatingMode);
        if (configureSuccess == true && inputConfig != null) {
            input = mRemoteDevice.getInputSurface();
        }
    } catch (CameraAccessException e) {
        ...
    }
    ...
    CameraCaptureSessionCore newSession = null;
    // 根据模式来实例化对应的 Session
    if (isConstrainedHighSpeed) {
        newSession = new CameraConstrainedHighSpeedCaptureSessionImpl(
            mNextSessionId++, callback, handler, this, 
            mDeviceHandler, configureSuccess, mCharacteristics);
    } else {
        // 假设实例化 CameraCaptureSessionImpl
        newSession = new CameraCaptureSessionImpl(mNextSessionId++, 
            input, callback, handler, this, 
            mDeviceHandler, configureSuccess);
    }

    mCurrentSession = newSession;
    ...
}
  • 将 List<Surface> 转换为 List<OutputConfiguration>
  • createCaptureSession 创建会话时，输入 Surface, InputConfiguration 都为空，即只有输出流
  • 根据 isConstrainedHighSpeed 来创建 CameraCaptureSession 实例；如果支持高速模式，则创建 CameraConstrainedHighSpeedCaptureSessionImpl 实例；否则创建普通 CameraCaptureSessionImpl 实例

示例

创建预览 mTextureSurface 和拍照 ImageReader.getSurface 两个输出流的会话，使用当前线程处理回调接口： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
mCameraDevice.createCaptureSession(
    Arrays.asList(mTextureSurface, mImageReader.getSurface()), 
    new CameraCaptureSession.StateCallback() {
        @Override
        public void onConfigured(@NonNull CameraCaptureSession session) {
            Log.d(TAG, "onConfigured: ");
            mCameraCaptureSession = session;
            preview();
        }

        @Override
        public void onConfigureFailed(
            @NonNull CameraCaptureSession session) {
            Log.e(TAG, "onConfigureFailed: ");
        }
    }, 
    null);
} catch (CameraAccessException e) {
    e.printStackTrace();
}

CameraCaptureSession.StateCallback 回调

CameraCaptureSession.StateCallback 回调用来处理 createCaptureSession 创建会话过程中出现的各种状态，比如创建成功、失败等，这些回调处理直接在 API Java 层实现的；回调接口中会获取到 CameraCaptureSession 实例。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
public static abstract class StateCallback {
    public abstract void onConfigured(
        @NonNull CameraCaptureSession session);
    public abstract void onConfigureFailed(
        @NonNull CameraCaptureSession session);
    public void onReady(@NonNull CameraCaptureSession session) {
        // default empty implementation
    }
    public void onActive(@NonNull CameraCaptureSession session) {
        // default empty implementation
    }
    public void onCaptureQueueEmpty(
        @NonNull CameraCaptureSession session) {
        // default empty implementation
    }
    public void onClosed(@NonNull CameraCaptureSession session) {
        // default empty implementation
    }
    public void onSurfacePrepared(@NonNull CameraCaptureSession session,
        @NonNull Surface surface) {
        // default empty implementation
    }
}

createCaptureSession 创建会话时，会创建 CameraCaptureSessionImpl 实例，而 CameraCaptureSession.StateCallback 接口的回调都是在 CameraCaptureSessionImpl 中实现的： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
// CameraCaptureSessionImpl.java
public class CameraCaptureSessionImpl extends CameraCaptureSession
        implements CameraCaptureSessionCore {
    CameraCaptureSessionImpl(int id, Surface input,
            CameraCaptureSession.StateCallback callback,
            Handler stateHandler,
            android.hardware.camera2.impl.CameraDeviceImpl deviceImpl,
            Handler deviceStateHandler, boolean configureSuccess) {
        ...
        mStateCallback = 
            createUserStateCallbackProxy(mStateHandler, callback);
        ...
        // 根据传入的参数，响应 CameraCaptureSession.StateCallback 回调
        if (configureSuccess) {
            mStateCallback.onConfigured(this);
            if (DEBUG) Log.v(...);
            mConfigureSuccess = true;
        } else {
            mStateCallback.onConfigureFailed(this);
            mClosed = true; 
            Log.e(...);
            mConfigureSuccess = false;
        }
    }
    ...
    
    // 用户指定并传入的回调实现及对应线程
    private final CameraCaptureSession.StateCallback mStateCallback;
    private final Handler mStateHandler;
    
    private StateCallback createUserStateCallbackProxy(Handler handler,
        StateCallback callback) {
        InvokeDispatcher<StateCallback> userCallbackSink 
            = new InvokeDispatcher<>(callback);
        HandlerDispatcher<StateCallback> handlerPassthrough =
                new HandlerDispatcher<>(userCallbackSink, handler);
        // 创建代理类
        return new CallbackProxies.SessionStateCallbackProxy(
            handlerPassthrough);
    }
}

用户指定的 StateCallback 传入后，在方法 createUserStateCallbackProxy 中，通过 CallbackProxies 重新生成一个代理 mStateCallback 对象，通过反射的方式，完成所有回调响应过程。

  • 如果 configureStreamsChecked 创建 Stream 成功，则响应回调 mStateCallback.onConfigured
  • 如果失败则响应 mStateCallback.onConfigureFailed ，其他场景会产生剩余的回调

动态代理类 CallbackProxies 源码注释（JDK 中的动态代理只支持接口 interface，对于抽象类只能自己实现了）： 

1
2
3
4
5
6
7
8
9
10
/**
 * Proxy out invocations to the camera2 API callbacks into 
 * a {@link Dispatchable}.
 *
 * <p>Since abstract classes do not support Java's dynamic {@code Proxy},
 * we have to to use our own proxy mechanism.</p>
 */
public class CallbackProxies {

}

流程图

创建会话流程，查看原图

0109-android-camera-4-framework-CameraService-addService-CameraDevice_createCaptureSession.jpg

源码分析

API 中创建捕获会话 createCaptureSession 时，CameraDeviceImpl.configureStreamsChecked 源码中可以看到；CameraDeviceImpl 是通过 ICameraDeviceUser 来向 Framework, HAL 层发送配置信息的： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
// CameraDeviceImpl.java
public boolean configureStreamsChecked(InputConfiguration inputConfig,
        List<OutputConfiguration> outputs, int operatingMode)
                throws CameraAccessException {
    ...
    // createCaptureSession 时，imputConfig 为空
    checkInputConfiguration(inputConfig);
    boolean success = false;
    synchronized(mInterfaceLock) {
        ...
        mDeviceHandler.post(mCallOnBusy);
        stopRepeating();

        try {
            waitUntilIdle();
            mRemoteDevice.beginConfigure();
            ...
            // Delete all streams first (to free up HW resources)
            for (Integer streamId : deleteList) {
                mRemoteDevice.deleteStream(streamId);
                mConfiguredOutputs.delete(streamId);
            }

            // Add all new streams
            for (OutputConfiguration outConfig : outputs) {
                if (addSet.contains(outConfig)) {
                    int streamId = mRemoteDevice.createStream(outConfig);
                    mConfiguredOutputs.put(streamId, outConfig);
                }
            }
            operatingMode = (operatingMode | (customOpMode << 16));
            mRemoteDevice.endConfigure(operatingMode);

            success = true;
        } catch (...)
    }

    return success;
}

configureStreamsChecked 配置流有三个主要过程：beginConfigure, createStream, endConfigure ，都是通过 ICameraDeviceUser 向下发送信息。 native 代码中由 CameraDeviceClient.cpp 实现了 ICameraDeviceUser 中的所有功能，这里重点分析 createStream 函数： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
// CameraDeviceClient.cpp
binder::Status CameraDeviceClient::createStream(
        const hardware::camera2::params::OutputConfiguration 
        &outputConfiguration,
        /*out*/int32_t* newStreamId) {
    ...
    // 获取 IGraphicBufferProducer 的个数
    const std::vector<sp<IGraphicBufferProducer>>& bufferProducers =
            outputConfiguration.getGraphicBufferProducers();
    size_t numBufferProducers = bufferProducers.size();
    ...
    std::vector<sp<Surface>> surfaces;
    std::vector<sp<IBinder>> binders;
    ...
    OutputStreamInfo streamInfo;
    bool isStreamInfoValid = false;
    for (auto& bufferProducer : bufferProducers) {
        ...
        // 创建 Native Surface
        sp<Surface> surface;
        res = createSurfaceFromGbp(streamInfo, isStreamInfoValid, 
            surface, bufferProducer);
        if (!res.isOk())
            return res;
        ...
        binders.push_back(IInterface::asBinder(bufferProducer));
        surfaces.push_back(surface);
    }

    int streamId = camera3::CAMERA3_STREAM_ID_INVALID;
    // 调用 Camera3Device 创建流
    err = mDevice->createStream(surfaces, deferredConsumer, 
            streamInfo.width, streamInfo.height, 
            streamInfo.format, streamInfo.dataSpace,
            static_cast<camera3_stream_rotation_t>(
                outputConfiguration.getRotation()),
            &streamId, outputConfiguration.getSurfaceSetID(), isShared);
            
    if (err != OK) {
        ...
    } else {
        ...
        mStreamInfoMap[streamId] = streamInfo;
        ...
        // Set transform flags to ensure preview to be rotated correctly.
        res = setStreamTransformLocked(streamId);
        *newStreamId = streamId;
    }

    return res;
}

CameraDeviceClient.createStream 中，将 API 传入的 OutputConfiguration 数据，转换成 native Surface, OutputStreamInfo ；根据 OutputConfiguration 中 IGraphicBufferProducer 的个数创建对应的 native Surface ，并最终通过设备来创建流。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
binder::Status CameraDeviceClient::createSurfaceFromGbp(
        OutputStreamInfo& streamInfo, bool isStreamInfoValid,
        sp<Surface>& surface, const sp<IGraphicBufferProducer>& gbp) {

    ...
    // 根据 IGraphicBufferProducer 创建 Surface
    surface = new Surface(gbp, useAsync);
    ANativeWindow *anw = surface.get();

    int width, height, format;
    android_dataspace dataSpace;
    // 查询对应的长、宽、格式、数据空间
    if ((err = anw->query(anw, NATIVE_WINDOW_WIDTH, &width)) != OK) {
        ...
    }
    if ((err = anw->query(anw, NATIVE_WINDOW_HEIGHT, &height)) != OK) {
        ...
    }
    if ((err = anw->query(anw, NATIVE_WINDOW_FORMAT, &format)) != OK) {
        ...
    }
    if ((err = anw->query(anw, NATIVE_WINDOW_DEFAULT_DATASPACE,
            reinterpret_cast<int*>(&dataSpace))) != OK) {
        ...
    }

    ...
    // 赋值给输出流
    if (!isStreamInfoValid) {
        streamInfo.width = width;
        streamInfo.height = height;
        streamInfo.format = format;
        streamInfo.dataSpace = dataSpace;
        streamInfo.consumerUsage = consumerUsage;
        return binder::Status::ok();
    }
    ...
}

这里的 NATIVE_WINDOW_FORMAT 格式代表着不同流的类型，在 system\core\libsystem\include\system\graphics-base.h 文件中定义： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
typedef enum {
    HAL_PIXEL_FORMAT_RGBA_8888 = 1,
    HAL_PIXEL_FORMAT_RGBX_8888 = 2,
    HAL_PIXEL_FORMAT_RGB_888 = 3,
    HAL_PIXEL_FORMAT_RGB_565 = 4,
    HAL_PIXEL_FORMAT_BGRA_8888 = 5,
    HAL_PIXEL_FORMAT_RGBA_1010102 = 43, // 0x2B
    HAL_PIXEL_FORMAT_RGBA_FP16 = 22, // 0x16
    HAL_PIXEL_FORMAT_YV12 = 842094169, // 0x32315659
    HAL_PIXEL_FORMAT_Y8 = 538982489, // 0x20203859
    HAL_PIXEL_FORMAT_Y16 = 540422489, // 0x20363159
    HAL_PIXEL_FORMAT_RAW16 = 32, // 0x20
    HAL_PIXEL_FORMAT_RAW10 = 37, // 0x25
    HAL_PIXEL_FORMAT_RAW12 = 38, // 0x26
    HAL_PIXEL_FORMAT_RAW_OPAQUE = 36, // 0x24
    HAL_PIXEL_FORMAT_BLOB = 33, // 0x21
    HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED = 34, // 0x22
    HAL_PIXEL_FORMAT_YCBCR_420_888 = 35, // 0x23
    HAL_PIXEL_FORMAT_YCBCR_422_888 = 39, // 0x27
    HAL_PIXEL_FORMAT_YCBCR_444_888 = 40, // 0x28
    HAL_PIXEL_FORMAT_FLEX_RGB_888 = 41, // 0x29
    HAL_PIXEL_FORMAT_FLEX_RGBA_8888 = 42, // 0x2A
    HAL_PIXEL_FORMAT_YCBCR_422_SP = 16, // 0x10
    HAL_PIXEL_FORMAT_YCRCB_420_SP = 17, // 0x11
    HAL_PIXEL_FORMAT_YCBCR_422_I = 20, // 0x14
    HAL_PIXEL_FORMAT_JPEG = 256, // 0x100
} android_pixel_format_t;
  • HAL_PIXEL_FORMAT_BLOB 拍照流
    值为 33 ，通常对应 mImageReader.getSurface()
  • HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED 预览和录像流
    值为 34 ，通常对应预览 new Surface(mTextureView.getSurfaceTexture()) 和录像 mMediaRecorder.getSurface 。

CameraDeviceClient::createStream 最终会调用 Camera3Device::createStream ，它会根据 NATIVE_WINDOW_FORMAT 格式创建不同配置的流： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
// Camera3Device.cpp
status_t Camera3Device::createStream(
        const std::vector<sp<Surface>>& consumers,
        bool hasDeferredConsumer, uint32_t width, uint32_t height, 
        int format, android_dataspace dataSpace, 
        camera3_stream_rotation_t rotation, int *id,
        int streamSetId, bool isShared, uint64_t consumerUsage) {
    ...
    // 拍照流
    if (format == HAL_PIXEL_FORMAT_BLOB) {
        ssize_t blobBufferSize;
        if (dataSpace != HAL_DATASPACE_DEPTH) {
            blobBufferSize = getJpegBufferSize(width, height);
            if (blobBufferSize <= 0) {
                SET_ERR_L(...);
                return BAD_VALUE;
            }
        } else {
            blobBufferSize = getPointCloudBufferSize();
            if (blobBufferSize <= 0) {
                SET_ERR_L(...);
                return BAD_VALUE;
            }
        }
        newStream = new Camera3OutputStream(mNextStreamId, consumers[0],
                width, height, blobBufferSize, format, dataSpace, 
                rotation, mTimestampOffset, streamSetId);
    } else if (format == HAL_PIXEL_FORMAT_RAW_OPAQUE) {
        ssize_t rawOpaqueBufferSize = 
            getRawOpaqueBufferSize(width, height);
        if (rawOpaqueBufferSize <= 0) {
            SET_ERR_L(...);
            return BAD_VALUE;
        }
        newStream = new Camera3OutputStream(mNextStreamId, consumers[0],
                width, height, rawOpaqueBufferSize, format, dataSpace, 
                rotation, mTimestampOffset, streamSetId);
    }
    ...
    else {
        newStream = new Camera3OutputStream(mNextStreamId, consumers[0],
                width, height, format, dataSpace, rotation,
                mTimestampOffset, streamSetId);
    }
    newStream->setStatusTracker(mStatusTracker);
    newStream->setBufferManager(mBufferManager);

    res = mOutputStreams.add(mNextStreamId, newStream);
    if (res < 0) {
        SET_ERR_L(...);
        return res;
    }

    *id = mNextStreamId++;
    mNeedConfig = true;

    ...
    ALOGV("Camera %s: Created new stream", mId.string());
    return OK;
}

注意：每配置一个输出 Surface ，都会创建对应的输出流 Camera3OutputStream ，这是一个 for 循环过程。
在 API 调用过程中，CameraDeviceImpl.configureStreamsChecked 的第三步为 endConfigure，而 CameraDeviceClient::endConfigure 代码流程如下： 

1
2
3
4
5
6
7
// CameraDeviceClient.cpp
binder::Status CameraDeviceClient::endConfigure(int operatingMode) {
    ...
    status_t err = mDevice->configureStreams(operatingMode);
    ...
    return res;
}

它的主要作用就是通过 Camera3Device 来配置流，configureStreamsLocked 配置流主要有三个过程：startConfiguration, configureStreams, endConfigure ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
// Camera3Device.cpp
status_t Camera3Device::configureStreams(int operatingMode) {
    ...
    return configureStreamsLocked(operatingMode);
}

status_t Camera3Device::configureStreamsLocked(int operatingMode) {
    ...
    camera3_stream_configuration config;
    config.operation_mode = mOperatingMode;
    config.num_streams = (mInputStream != NULL) + mOutputStreams.size();

    Vector<camera3_stream_t*> streams;
    streams.setCapacity(config.num_streams);

    if (mInputStream != NULL) {
        camera3_stream_t *inputStream;
        inputStream = mInputStream->startConfiguration();
        if (inputStream == NULL) {
            CLOGE("Can't start input stream configuration");
            cancelStreamsConfigurationLocked();
            return INVALID_OPERATION;
        }
        streams.add(inputStream);
    }

    for (size_t i = 0; i < mOutputStreams.size(); i++) {
        if (mOutputStreams[i].get() ==
            static_cast<Camera3StreamInterface*>(mInputStream.get())) {

            config.num_streams--;
            continue;
        }

        camera3_stream_t *outputStream;
      outputStream = mOutputStreams.editValueAt(i)->startConfiguration();
        if (outputStream == NULL) {
            CLOGE("Can't start output stream configuration");
            cancelStreamsConfigurationLocked();
            return INVALID_OPERATION;
        }
        streams.add(outputStream);
    }

    config.streams = streams.editArray();

    // Do the HAL configuration; will potentially touch stream
    // max_buffers, usage, priv fields.
    // 向 HAL 层发送配置信息
    res = mInterface->configureStreams(&config);
    ...
    if (mInputStream != NULL && mInputStream->isConfiguring()) {
        res = mInputStream->finishConfiguration();
        if (res != OK) {
            CLOGE("Can't finish configuring input stream %d: %s (%d)",
                    mInputStream->getId(), strerror(-res), res);
            cancelStreamsConfigurationLocked();
            return BAD_VALUE;
        }
    }

    for (size_t i = 0; i < mOutputStreams.size(); i++) {
        sp<Camera3OutputStreamInterface> outputStream =
            mOutputStreams.editValueAt(i);
        if (outputStream->isConfiguring() && 
            !outputStream->isConsumerConfigurationDeferred()) {
            res = outputStream->finishConfiguration();
            if (res != OK) {
                CLOGE(...);
                cancelStreamsConfigurationLocked();
                return BAD_VALUE;
            }
        }
    }
    ...
}

最终通过 Camera3Device::HalInterface::configureStreams 向 HAL 层发起配置信息： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
// Camera3Device.cpp
status_t Camera3Device::HalInterface::configureStreams(
    camera3_stream_configuration *config) {
    ...
    // Invoke configureStreams

    device::V3_3::HalStreamConfiguration finalConfiguration;
    common::V1_0::Status status;

    // See if we have v3.3 HAL
    sp<device::V3_3::ICameraDeviceSession> hidlSession_3_3;
    auto castResult = 
        device::V3_3::ICameraDeviceSession::castFrom(mHidlSession);
    if (castResult.isOk()) {
        hidlSession_3_3 = castResult;
    } else {
        ALOGE(...);
    }
    if (hidlSession_3_3 != nullptr) {
        // We do; use v3.3 for the call
        ALOGV("%s: v3.3 device found", __FUNCTION__);
        auto err = hidlSession_3_3->configureStreams_3_3(
            requestedConfiguration,
            [&status, &finalConfiguration]
            (common::V1_0::Status s, const 
                device::V3_3::HalStreamConfiguration& halConfiguration) {
                finalConfiguration = halConfiguration;
                status = s;
            });
        if (!err.isOk()) {
            ALOGE(...);
            return DEAD_OBJECT;
        }
    } else {
        // We don't; use v3.2 call and 
        // construct a v3.3 HalStreamConfiguration
        ALOGV("%s: v3.2 device found", __FUNCTION__);
        HalStreamConfiguration finalConfiguration_3_2;
        auto err = mHidlSession->configureStreams(requestedConfiguration,
                [&status, &finalConfiguration_3_2]
                (common::V1_0::Status s, const 
                    HalStreamConfiguration& halConfiguration) {
                    finalConfiguration_3_2 = halConfiguration;
                    status = s;
                });
        if (!err.isOk()) {
            ALOGE(...);
            return DEAD_OBJECT;
        }
        finalConfiguration.streams.resize(
            finalConfiguration_3_2.streams.size());
        for (size_t i = 0; i<finalConfiguration_3_2.streams.size(); i++){
            finalConfiguration.streams[i].v3_2 = 
                finalConfiguration_3_2.streams[i];
            finalConfiguration.streams[i].overrideDataSpace =
                    requestedConfiguration.streams[i].dataSpace;
        }
    }
    ...
}

这里需要注意的是 HAL 3.3, 3.2 的配置是有区别的；执行完配置后，Camera3Stream::finishConfiguration 结束配置： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
// Camera3Stream.cpp
status_t Camera3Stream::finishConfiguration() {
    ....
    res = configureQueueLocked();
    ...
}

// Camera3OutputStream.cpp
status_t Camera3OutputStream::configureConsumerQueueLocked() {
    ...
    // Configure consumer-side ANativeWindow interface. 
    // to notify buffer manager (if it is used) of the returned buffers.
    res = mConsumer->connect(NATIVE_WINDOW_API_CAMERA,
            /*listener*/mBufferReleasedListener,
            /*reportBufferRemoval*/true);
    if (res != OK) {
        ALOGE(...);
        return res;
    }

    mConsumerName = mConsumer->getConsumerName();

    res = native_window_set_usage(mConsumer.get(), mUsage);
    if (res != OK) {
        ALOGE(...);
        return res;
    }

    res = native_window_set_scaling_mode(mConsumer.get(),
            NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW);
    if (res != OK) {
        ALOGE(...);
        return res;
    }
    ...
}

这里 mConsumer 是 native Surface，也就是将生产者-消费者模型连接起来；同时 configureConsumerQueueLocked 有非常多的 native window 配置。

小结

创建会话 createCaptureSession 过程中，小结如下：

  • API 调用时，最终通过 CameraCaptureSession.StateCallback 获取到 CameraCaptureSessionImpl 实例
  • ICameraDeviceUser.createStream 由输入的 Surface 信息，根据不同的 format 创建对应输出流 Camera3OutputStream
  • ICameraDeviceUser.endConfigure 最终通过 CameraDeviceSession.configureStream_3_3 会向 HAL 层发送配置信息

相机预览过程中，如果 session 创建成功，会出现正常的预览界面；如果 session 创建失败，则预览会出现黑屏。

预览/拍照/录像流程

API

创建会话 createCaptureSession 成功后，通过拿到的 CameraCaptureSession 来预览、拍照、录像： 

1
2
3
4
5
6
7
8
9
10
// CameraCaptureSession.java
// 预览和录像使用同一个 API
public abstract int setRepeatingRequest(@NonNull CaptureRequest request,
        @Nullable CaptureCallback listener, 
        @Nullable Handler handler)
        throws CameraAccessException;
// 拍照
public abstract int capture(@NonNull CaptureRequest request,
        @Nullable CaptureCallback listener, @Nullable Handler handler)
        throws CameraAccessException;
  • CaptureRequest request
    捕获请求，比如创建一个预览模板的请求 CameraDevice.TEMPLATE_PREVIEW ；拍照模板的请求 CameraDevice.TEMPLATE_STILL_CAPTURE ；录像模板的请求 CameraDevice.TEMPLATE_RECORD 。
  • CameraCaptureSession.CaptureCallback listener
    捕获状态的回调接口。
  • Handler handler
    回调接口使用哪个线程响应，如果是 null 表示当前线程。

CameraDevice 请求模板是一组常量： 

1
2
3
4
5
6
7
// CameraDevice.java
public static final int TEMPLATE_PREVIEW = 1;
public static final int TEMPLATE_STILL_CAPTURE = 2;
public static final int TEMPLATE_RECORD  = 3;
public static final int TEMPLATE_VIDEO_SNAPSHOT = 4;
public static final int TEMPLATE_ZERO_SHUTTER_LAG = 5;
public static final int TEMPLATE_MANUAL = 6;

各模板对应的含义：

  • TEMPLATE_PREVIEW
    创建适合相机预览的窗口，高帧率优于高质量的后期处理。
  • TEMPLATE_STILL_CAPTURE
    创建适合拍照的请求，优先考虑帧速率的图像质量。
  • TEMPLATE_RECORD
    创建适合录像的请求，使用稳定的帧率。
  • TEMPLATE_VIDEO_SNAPSHOT
    创建录像时快照的请求，在不中断录像的前提下最大化图像质量。
  • TEMPLATE_ZERO_SHUTTER_LAG
    创建 ZSL 零延时拍照请求，也就是连拍功能，在不影响帧率的前提下最大化图像质量，并开启 3A 算法。
  • TEMPLATE_MANUAL
    手动控制模板，禁用所有的自动控制 3A 算法。

示例

给 mTextureSurface 创建预览请求 TEMPLATE_PREVIEW ，使用后台线程处理回调接口： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
CaptureRequest.Builder previewRequestBuilder = 
    mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
previewRequestBuilder.addTarget(mTextureSurface);
CameraCaptureSession.CaptureCallback captureCallback 
    = new CameraCaptureSession.CaptureCallback() {
        @Override
        public void onCaptureCompleted(
            @NonNull CameraCaptureSession session, 
            @NonNull CaptureRequest request, 
            @NonNull TotalCaptureResult result) {
            //Log.d(TAG, "preview, onCaptureCompleted: ");
        }
};
mCameraCaptureSession.setRepeatingRequest(
    previewRequestBuilder.build(), captureCallback, mBackHandler);

给 ImageReader 创建拍照请求 TEMPLATE_STILL_CAPTURE ，使用后台线程处理回调： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
CaptureRequest.Builder captureRequestBuild = mCameraDevice
    .createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
captureRequestBuild.addTarget(mImageReader.getSurface());
captureRequestBuild.set(CaptureRequest.CONTROL_AF_MODE,
    CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
captureRequestBuild.set(CaptureRequest.JPEG_ORIENTATION, 
    mImageOrientation);
CameraCaptureSession.CaptureCallback captureCallback = 
    new CameraCaptureSession.CaptureCallback() {
        @Override
        public void onCaptureCompleted(
            @NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request,
            @NonNull TotalCaptureResult result) {
            // Log.d(TAG, "takePicture, onCaptureCompleted: ");
        }
};
mCameraCaptureSession.stopRepeating();
mCameraCaptureSession.abortCaptures();
mCameraCaptureSession.capture(captureRequestBuild.build(),
    captureCallback, mBackHandler);

给 MediaRecorder 创建录像请求 TEMPLATE_RECORD ，不处理回调： 

1
2
3
4
5
6
7
CaptureRequest.Builder recordCaptureBuild = mCameraDevice
    .createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
recordCaptureBuild.addTarget(mMediaRecorder.getSurface());
recordCaptureBuild.set(CaptureRequest.CONTROL_MODE, 
    CameraMetadata.CONTROL_MODE_AUTO);
mCameraCaptureSession.setRepeatingRequest(
    recordCaptureBuild.build(), null, null);

CameraCaptureSession.CaptureCallback 回调

CameraCaptureSession 在请求预览、拍照、录像等功能时，出现的各种状态通过 CameraCaptureSession.CaptureCallback 回调来处理，回调是由 HAL 层发起向上传递的；回调接口中通常包含当前会话信息 CameraCaptureSession ，捕获请求 CaptureRequest ，捕获的结果 CaptureResult 等。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
// CameraCaptureSession.java
public static abstract class CaptureCallback {

    public static final int NO_FRAMES_CAPTURED = -1;

    // 设备开始拍照时
    public void onCaptureStarted(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, long timestamp, 
            long frameNumber) {
        // default empty implementation
    }
    // 返回部分数据
    public void onCapturePartial(CameraCaptureSession session,
            CaptureRequest request, CaptureResult result) {
        // default empty implementation
    }
    // 返回部分数据时，表示正在捕获数据过程中
   public void onCaptureProgressed(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, 
            @NonNull CaptureResult partialResult) {
        // default empty implementation
    }
    // 数据捕获已经完成，回调最终总的结果集
    public void onCaptureCompleted(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, 
            @NonNull TotalCaptureResult result) {
        // default empty implementation
    }
    // 捕获失败
    public void onCaptureFailed(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, 
            @NonNull CaptureFailure failure) {
        // default empty implementation
    }
    // 所有捕获结果已经发送完毕
    public void onCaptureSequenceCompleted(
            @NonNull CameraCaptureSession session,
            int sequenceId, long frameNumber) {
        // default empty implementation
    }
    // 请求捕获被中止
    public void onCaptureSequenceAborted(
            @NonNull CameraCaptureSession session,
            int sequenceId) {
        // default empty implementation
    }
    // 捕获的 buffer 不能成功显示
   public void onCaptureBufferLost(@NonNull CameraCaptureSession session,
            @NonNull CaptureRequest request, 
            @NonNull Surface target, long frameNumber) {
        // default empty implementation
    }
}

API 在发起请求 CameraCaptureSession.setRepeatingRequest/capture 时，用户会创建 CaptureCallback 的实例，这些接口都在 CameraDeviceImpl 中实现回调。

  • onCaptureStarted
    该回调接口从 HAL 回调路径为：CameraDeviceClient::notifyShutter -> CameraDeviceImpl.onCaptureStarted -> onCaptureStarted 。
  • onCapturePartial
    该回调接口搜索整个 framework ，发现没有任何地方会回调它。
  • onCaptureProgressed, onCaptureCompleted, onCaptureSequenceCompleted
    三个接口都是在 CameraDeviceImpl.onResultReceived 中回调的。
  • onCaptureSequenceAborted
    CameraDeviceClient.cpp 中的 submitCaptureRequest, stopRepeating, flush 这三个函数会回调该接口。
  • onCaptureFailed, onCaptureBufferLost
    从 HAL 回调路径为 CameraDeviceClient::notifyError -> CameraDeviceImpl.onDeviceError ，而这两个接口在 CameraDeviceImpl 中的回调路径为 onDeviceError -> onCaptureErrorLocked -> onCaptureFailed/onCaptureBufferLost 。

流程图

创建捕获请求流程，查看原图

0109-android-camera-4-framework-CameraDevice_createCaptureRequest.jpg

预览/拍照/录像流程基本一致，这里仅给出预览的流程图：预览流程，查看原图

0109-android-camera-4-framework-CameraCaptureSession_setRepeatingRequest.jpg

源码分析

在分析预览、拍照、录像流程前，先回顾下打开设备 openCamera 时，做的一些初始化： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// CameraDeviceClient.cpp
status_t CameraDeviceClient::initialize(sp<CameraProviderManager> manager) {
    return initializeImpl(manager);
}

template<typename TProviderPtr>
status_t CameraDeviceClient::initializeImpl(TProviderPtr providerPtr) {
    ...
    res = Camera2ClientBase::initialize(providerPtr);
    if (res != OK) {
        return res;
    }

    String8 threadName;
    mFrameProcessor = new FrameProcessorBase(mDevice);
    threadName = String8::format("CDU-%s-FrameProc", 
        mCameraIdStr.string());
    mFrameProcessor->run(threadName.string());

    mFrameProcessor->registerListener(FRAME_PROCESSOR_LISTENER_MIN_ID,
                                      FRAME_PROCESSOR_LISTENER_MAX_ID,
                                      /*listener*/this,
                                      /*sendPartials*/true);

    return OK;
}

在 CameraDeviceClient::initializeImpl 中，调用了 Camera2ClientBase::initialize 的初始化，以及实例化一个 FrameProcessorBase 对象； 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Camera2ClientBase.cpp
template <typename TClientBase>
status_t Camera2ClientBase<TClientBase>::initialize(
        sp<CameraProviderManager> manager) {
    return initializeImpl(manager);
}

template <typename TClientBase>
template <typename TProviderPtr>
status_t Camera2ClientBase<TClientBase>::initializeImpl(
        TProviderPtr providerPtr) {
    ...
    res = mDevice->initialize(providerPtr);
    ...
}

这里 Camera2ClientBase::initializeImpl 中主要是调用了 Camera3Device::initialize 函数，下面只关心和捕获请求有关的代码： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Camera3Device.cpp
status_t Camera3Device::initialize(sp<CameraProviderManager> manager) {
    ...
    return initializeCommonLocked();
}

status_t Camera3Device::initializeCommonLocked() {
    ...
    /** Start up request queue thread */
    mRequestThread = new RequestThread(this, mStatusTracker, mInterface);
    res = mRequestThread->run(
        String8::format("C3Dev-%s-ReqQueue", mId.string()).string());
    ...
    return OK;
}

在 Camera3Device::initializeCommonLocked 中实例化了 RequestThread 对象。至此，捕获流程中的发起请求的对象 RequestThread 和响应回调的对象 FrameProcessorBase 都实例化完毕，并开始运行。他们两个都继承的是线程，参看 system 目录下的 Thread.h/Threads.cpp 源码，可以看到 threadLoop 是在一个 while 中被循环调用的。当 threadLoop 返回 true 时就会不停的循环；返回 false 时会退出循环： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
// Threads.cpp
int Thread::_threadLoop(void* user)
{
    Thread* const self = static_cast<Thread*>(user);
    ...
    bool first = true;
    do {
        bool result;
        if (first) {
            first = false;
            self->mStatus = self->readyToRun();
            result = (self->mStatus == NO_ERROR);

            if (result && !self->exitPending()) {
                result = self->threadLoop();
            }
        } else {
            result = self->threadLoop();
        }

        ...
        if (result == false || self->mExitPending) {
            ...
            break;
        }
        ...
    } while(strong != 0);

    return 0;
}

先来看发送捕获请求的线程 RequestThread ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
// Camera3Device.h
class RequestThread : public Thread {
  ...
  protected:
    ...
    virtual bool threadLoop();
  private:
    ...
    // Used to prepare a batch of requests.
    struct NextRequest {
        sp<CaptureRequest>              captureRequest;
        camera3_capture_request_t       halRequest;
        Vector<camera3_stream_buffer_t> outputBuffers;
        bool                            submitted;
    };
    Vector<NextRequest> mNextRequests;
    ...
    Condition          mRequestSignal;
    RequestList        mRequestQueue;
    RequestList        mRepeatingRequests;
    ...
}

这里只关注 RequestThread 类中几个关键函数和变量，NextRequest 结构体包含了请求信息，逐个向 HAL 发送这些信息；类中定义了多个条件变量，重点关注 mRequestSignal 条件变量， threadLoop 运行时，会通过 mRequestSignal.waitRelative 阻塞等待 50 ms；直到等到捕获请求后 mRequestSignal.signal 发出通知，threadLoop 继续运行。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
// Camera3Device.cpp
bool Camera3Device::RequestThread::threadLoop() {
    ...
    status_t res;

    // Handle paused state.
    if (waitIfPaused()) {
        return true;
    }

    // Wait for the next batch of requests.
    waitForNextRequestBatch();
    if (mNextRequests.size() == 0) {
        return true;
    }
    ...
    // Prepare a batch of HAL requests and output buffers.
    res = prepareHalRequests();
    ...
}

void Camera3Device::RequestThread::waitForNextRequestBatch() {
    ...
    NextRequest nextRequest;
    nextRequest.captureRequest = waitForNextRequestLocked();
    if (nextRequest.captureRequest == nullptr) {
        return;
    }
    ...
}

sp<Camera3Device::CaptureRequest>
        Camera3Device::RequestThread::waitForNextRequestLocked() {
    status_t res;
    sp<CaptureRequest> nextRequest;

    while (mRequestQueue.empty()) {
        if (!mRepeatingRequests.empty()) {
            const RequestList &requests = mRepeatingRequests;
            RequestList::const_iterator firstRequest =
                    requests.begin();
            nextRequest = *firstRequest;
            mRequestQueue.insert(mRequestQueue.end(),
                    ++firstRequest,
                    requests.end());

            mRepeatingLastFrameNumber = mFrameNumber+requests.size()-1;

            break;
        }
        // 条件变量 mRequestSignal 阻塞等待 kRequestTimeout 
        res = mRequestSignal.waitRelative(mRequestLock, kRequestTimeout);
        ...
    }
}

RequestThread 在没有捕获请求时，会循环调用 threadLoop ，并阻塞等待 mRequestSignal 的通知。再看响应回调的线程 FrameProcessorBase ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
// FrameProcessorBase.h
class FrameProcessorBase: public Thread {
  public:
    ...
    struct FilteredListener: virtual public RefBase {
        virtual void onResultAvailable(const CaptureResult &result) = 0;
    };
  protected:
    static const nsecs_t kWaitDuration = 10000000; // 10 ms
    wp<CameraDeviceBase> mDevice;
    virtual bool threadLoop();
  ...
}

// FrameProcessorBase.cpp
bool FrameProcessorBase::threadLoop() {
    status_t res;

    sp<CameraDeviceBase> device;
    {
        device = mDevice.promote();
        if (device == 0) return false;
    }

    res = device->waitForNextFrame(kWaitDuration);
    if (res == OK) {
        processNewFrames(device);
    } else if (res != TIMED_OUT) {
        ALOGE(...);
    }

    return true;
}

FrameProcessorBase::threadLoop 代码非常简单，device->waitForNextFrame 阻塞等待 10ms ，这里 CameraDeviceBase 实际类型为 Camera3Device ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Camera3Device.h
class Camera3Device :
    public CameraDeviceBase,
    virtual public hardware::camera::device::V3_2::ICameraDeviceCallback,
    private camera3_callback_ops {
    ...
    private:
        List<CaptureResult>   mResultQueue;
        Condition              mResultSignal;
    ...
}

// Camera3Device.cpp
status_t Camera3Device::waitForNextFrame(nsecs_t timeout) {
    ...
    while (mResultQueue.empty()) {
        res = mResultSignal.waitRelative(mOutputLock, timeout);
        if (res == TIMED_OUT) {
            return res;
        } else if (res != OK) {
            ALOGW(...);
            return res;
        }
    }
    return OK;
}

Camera3Device::waitForNextFrame 代码也很简单，调用条件变量 mResultSignal.waitRelative 实现阻塞等待 10 ms。
至此初始化过程中，捕获请求线程 RequestThread 循环执行 threadLoop ，并会阻塞等待 mRequestSignal 的通知；回调响应线程 FrameProcessorBase 循环执行 threadLoop ，并会阻塞等待 mResultSignal 的通知。

当用户调用 API 创建捕获请求时，mRequestSignal 会发出通知；因为预览、拍照、录像流程基本一样，一起分析： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
// CameraCaptureSessionImpl.java
// 预览和录像
@Override
public int setRepeatingRequest(CaptureRequest request, 
        CaptureCallback callback,
        Handler handler) 
        throws CameraAccessException {
    ...
    synchronized (mDeviceImpl.mInterfaceLock) {
        ...
        return addPendingSequence(mDeviceImpl.setRepeatingRequest(
            request,
            createCaptureCallbackProxy(handler, callback), 
            mDeviceHandler));
    }
}

// 拍照
@Override
public int capture(CaptureRequest request, CaptureCallback callback,
        Handler handler) throws CameraAccessException {
    ...
    synchronized (mDeviceImpl.mInterfaceLock) {
        ...
        return addPendingSequence(mDeviceImpl.capture(request,
                createCaptureCallbackProxy(handler, callback), 
                mDeviceHandler));
    }
}

createCaptureCallbackProxy 创建了一个回调动态代理，通过 CameraDeviceImpl.setRepeatingRequest/capture 下发预览或者拍照的捕获请求： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
// CameraDeviceImpl.java
// 预览和录像
public int setRepeatingRequest(CaptureRequest request, 
        CaptureCallback callback,
        Handler handler) throws CameraAccessException {
    List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
    requestList.add(request);
    return submitCaptureRequest(requestList, callback, 
        handler, /*streaming*/true);
}
// 拍照
public int capture(CaptureRequest request, CaptureCallback callback, 
        Handler handler) throws CameraAccessException {
    List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
    requestList.add(request);
    return submitCaptureRequest(requestList, callback, 
        handler, /*streaming*/false);
}

private int submitCaptureRequest(List<CaptureRequest> requestList, 
        CaptureCallback callback,
        Handler handler, boolean repeating) 
        throws CameraAccessException {
    ...
    synchronized(mInterfaceLock) {
        checkIfCameraClosedOrInError();
        if (repeating) {
            stopRepeating();
        }
        SubmitInfo requestInfo;
        CaptureRequest[] requestArray = requestList.toArray(
            new CaptureRequest[requestList.size()]);
        requestInfo = mRemoteDevice.submitRequestList(requestArray,
            repeating);
        ...
        if (callback != null) {
            mCaptureCallbackMap.put(requestInfo.getRequestId(),
                    new CaptureCallbackHolder(
                        callback, requestList, handler, 
                        repeating, mNextSessionId - 1));
        } else {
            ...
        }

        if (repeating) {
            if (mRepeatingRequestId != REQUEST_ID_NONE) {
                checkEarlyTriggerSequenceComplete(mRepeatingRequestId,
                        requestInfo.getLastFrameNumber());
            }
            mRepeatingRequestId = requestInfo.getRequestId();
        } else {
            mRequestLastFrameNumbersList.add(
                new RequestLastFrameNumbersHolder(requestList,
                    requestInfo));
        }
        ...
    }
}

从代码流程来看，预览和录像使用同一个接口；预览和拍照的主要区别是 repeating 的值；当为 true 时，表示预览/录像；当为 false 时，表示为拍照。通过 ICameraDeviceUser.submitRequestList 向下发送请求： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// CameraDeviceClient.cpp
binder::Status CameraDeviceClient::submitRequestList(
        const std::vector<hardware::camera2::CaptureRequest>& requests,
        bool streaming,
        /*out*/
        hardware::camera2::utils::SubmitInfo *submitInfo) {
    ...
    if (streaming) {
        err = mDevice->setStreamingRequestList(metadataRequestList, 
                surfaceMapList, &(submitInfo->mLastFrameNumber));
        ...
    } else {
        err = mDevice->captureList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
        ...
    }
    return res;
}

如果是预览/录像，则调用 Camera3Device->setStreamingRequestList ；如果是拍照，则调用 Camera3Device->captureList ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
//Camera3Device.cpp
// 预览和录像
status_t Camera3Device::setStreamingRequestList(
        const List<const CameraMetadata> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        int64_t *lastFrameNumber) {
    ...
    return submitRequestsHelper(requests, surfaceMaps, 
        /*repeating*/true, lastFrameNumber);
}
// 拍照
status_t Camera3Device::captureList(
        const List<const CameraMetadata> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        int64_t *lastFrameNumber) {
    ...
    return submitRequestsHelper(requests, surfaceMaps, 
        /*repeating*/false, lastFrameNumber);
}

status_t Camera3Device::submitRequestsHelper(
        const List<const CameraMetadata> &requests,
        const std::list<const SurfaceMap> &surfaceMaps,
        bool repeating, /*out*/ int64_t *lastFrameNumber) {
    ...
    RequestList requestList;
    res = convertMetadataListToRequestListLocked(requests, surfaceMaps,
            repeating, /*out*/&requestList);
    ...

    if (repeating) {
        res = mRequestThread->setRepeatingRequests(requestList,
            lastFrameNumber);
    } else {
        res = mRequestThread->queueRequestList(requestList,
            lastFrameNumber);
    }
    ...
    return res;
}

同样，预览/录像和拍照请求在 Camera3Device 中的区别也主要是 repeating 的值，都会调用 Camera3Device::submitRequestsHelper ，并通过 RequestThread 发起捕获请求；当预览/录像时，调用 setRepeatingRequests ；当拍照时，调用 queueRequestList ： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
// Camera3Device.cpp
// 预览和录像
status_t Camera3Device::RequestThread::setRepeatingRequests(
        const RequestList &requests,
        /*out*/
        int64_t *lastFrameNumber) {
    ...
    if (lastFrameNumber != NULL) {
        *lastFrameNumber = mRepeatingLastFrameNumber;
    }
    mRepeatingRequests.clear();
    mRepeatingRequests.insert(mRepeatingRequests.begin(),
            requests.begin(), requests.end());

    unpauseForNewRequests();

    mRepeatingLastFrameNumber = 
        ...ICameraDeviceUser::NO_IN_FLIGHT_REPEATING_FRAMES;
    return OK;
}
// 拍照
status_t Camera3Device::RequestThread::queueRequestList(
        List<sp<CaptureRequest> > &requests,
        /*out*/
        int64_t *lastFrameNumber) {
    ...
    for (List<sp<CaptureRequest> >::iterator it = requests.begin();
        it != requests.end(); ++it) {
        mRequestQueue.push_back(*it);
    }
    if (lastFrameNumber != NULL) {
        *lastFrameNumber = mFrameNumber + mRequestQueue.size() - 1;
        ALOGV(...);
    }

    unpauseForNewRequests();
    return OK;
}

void Camera3Device::RequestThread::unpauseForNewRequests() {
    ...
    mRequestSignal.signal();
    ...
}

预览/录像时会将捕获请求存入 mRepeatingRequests 列表中；拍照时会将捕获请求存入 mRequestQueue 列表中；它们最终都会调用 unpauseForNewRequests ，而该函数的核心功能就是通过 mRequestSignal.signal 发出消息，通知在开启设备初始化过程中 waitForNextRequestLocked 的阻塞等待。我们重新进入 RequestThread::threadLoop 中，继续向下分析： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
// Camera3Device.cpp
bool Camera3Device::RequestThread::threadLoop() {
    ...
    status_t res;
    ...
    // Wait for the next batch of requests.
    // 阻塞等待
    waitForNextRequestBatch();
    if (mNextRequests.size() == 0) {
        return true;
    }
    ...
    bool submitRequestSuccess = false;
    nsecs_t tRequestStart = systemTime(SYSTEM_TIME_MONOTONIC);
    if (mInterface->supportBatchRequest()) {
        submitRequestSuccess = sendRequestsBatch();
    } else {
        submitRequestSuccess = sendRequestsOneByOne();
    }
    ...
    return submitRequestSuccess;
}

bool Camera3Device::RequestThread::sendRequestsBatch() {
    status_t res;
    size_t batchSize = mNextRequests.size();
    std::vector<camera3_capture_request_t*> requests(batchSize);
    uint32_t numRequestProcessed = 0;
    for (size_t i = 0; i < batchSize; i++) {
        requests[i] = &mNextRequests.editItemAt(i).halRequest;
    }

    ...
    res = mInterface->processBatchCaptureRequests(requests, 
        &numRequestProcessed);
    ...
}

status_t Camera3Device::HalInterface::processBatchCaptureRequests(
        std::vector<camera3_capture_request_t*>& requests,
        /*out*/uint32_t* numRequestProcessed) {
    ...
    hardware::hidl_vec<device::V3_2::CaptureRequest> captureRequests;
    size_t batchSize = requests.size();
    captureRequests.resize(batchSize);
    ...
    auto err = mHidlSession->processCaptureRequest(captureRequests, 
            cachesToRemove,
            [&status, &numRequestProcessed] (auto s, uint32_t n) {
                status = s;
                *numRequestProcessed = n;
            });
    ...
    return CameraProviderManager::mapToStatusT(status);
}

当 waitForNextRequestBatch 拿到请求通知后，会将捕获请求存入 mNextRequests 中，当前平台支持批量请求处理，sendRequestsBatch -> processBatchCaptureRequests 流程，向 HAL 层发送捕获请求 mHidlSession->processCaptureRequest ，至此捕获请求从 API 发送到 HAL 整个流程全部分析完毕。

当 HAL 拿到捕获的结果后，会从 ICameraDeviceSession.processCaptureResult 回调到 Framework 层： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
// Camera3Device.cpp
hardware::Return<void> Camera3Device::processCaptureResult(
    const hardware::hidl_vec<
            hardware::camera::device::V3_2::CaptureResult>& results) {
    ...
    for (const auto& result : results) {
        processOneCaptureResultLocked(result);
    }
    mProcessCaptureResultLock.unlock();
    return hardware::Void();
}

void Camera3Device::processOneCaptureResultLocked(
        const hardware::camera::device::V3_2::CaptureResult& result) {
    camera3_capture_result r;
    ...
    processCaptureResult(&r);
}

void Camera3Device::processCaptureResult(
        const camera3_capture_result *result) {
    ...
    if (result->partial_result != 0)
        request.resultExtras.partialResultCount = result->partial_result;

    // Check if this result carries only partial metadata
    if (mUsePartialResult && result->result != NULL) {
        ...
        if (isPartialResult && request.hasCallback) {
            // Send partial capture result
            sendPartialCaptureResult(result->result, request.resultExtras,
                    frameNumber);
        }
    }
    ...
    if (result->result != NULL && !isPartialResult) {
        if (shutterTimestamp == 0) {
            request.pendingMetadata = result->result;
            request.collectedPartialResult = collectedPartialResult;
        } else if (request.hasCallback) {
            CameraMetadata metadata;
            metadata = result->result;
            sendCaptureResult(metadata, request.resultExtras,
                collectedPartialResult, frameNumber,
                hasInputBufferInRequest);
        }
    }
    ...
}

void Camera3Device::sendPartialCaptureResult(
        const camera_metadata_t * partialResult,
        const CaptureResultExtras &resultExtras, uint32_t frameNumber) {
    ...
    CaptureResult captureResult;
    captureResult.mResultExtras = resultExtras;
    captureResult.mMetadata = partialResult;
    insertResultLocked(&captureResult, frameNumber);
}


void Camera3Device::sendCaptureResult(CameraMetadata &pendingMetadata,
        CaptureResultExtras &resultExtras,
        CameraMetadata &collectedPartialResult,
        uint32_t frameNumber,
        bool reprocess) {
    ...
    CaptureResult captureResult;
    captureResult.mResultExtras = resultExtras;
    captureResult.mMetadata = pendingMetadata;
    ...
    insertResultLocked(&captureResult, frameNumber);
}

void Camera3Device::insertResultLocked(CaptureResult *result,
        uint32_t frameNumber) {
    ...
    camera_metadata_t *meta = const_cast<camera_metadata_t *>(
            result->mMetadata.getAndLock());
    set_camera_metadata_vendor_id(meta, mVendorTagId);
    result->mMetadata.unlock(meta);
    ...
    // Valid result, insert into queue
    List<CaptureResult>::iterator queuedResult =
            mResultQueue.insert(mResultQueue.end(), 
                CaptureResult(*result));
    ...
    mResultSignal.signal();
}

从代码流程来看，从 HAL 传过来的捕获结果，不管是发回部分结果 sendPartialCaptureResult 还是最终结果 sendCaptureResult ，最终都会调用 insertResultLocked ，它的主要功能就是将捕获结果放入 mResultQueue 队列，并由 mResultSignal.signal 发出消息，通知在开启设备初始化过程中 waitForNextFrame 的阻塞等待。一旦 FrameProcessorBase::threadLoop 获取到捕获结果后，逐个处理： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
// FrameProcessorBase.cpp
void FrameProcessorBase::processNewFrames(
        const sp<CameraDeviceBase> &device) {
    status_t res;
    ...
    while ( (res = device->getNextResult(&result)) == OK) {
        camera_metadata_entry_t entry;
        entry = result.mMetadata.find(ANDROID_REQUEST_FRAME_COUNT);
        ...
        if (!processSingleFrame(result, device)) {
            break;
        }
        ...
    }
    ...
    return;
}

bool FrameProcessorBase::processSingleFrame(CaptureResult &result,
         const sp<CameraDeviceBase> &device) {
    ...
    return processListeners(result, device) == OK;
}

status_t FrameProcessorBase::processListeners(
        const CaptureResult &result,
        const sp<CameraDeviceBase> &device) {
    ...
    List<sp<FilteredListener> >::iterator item = listeners.begin();
    for (; item != listeners.end(); item++) {
        (*item)->onResultAvailable(result);
    }
    return OK;
}

代理流程可以看出，逐个取出 CaptureResult 并处理，最终调用 CameraDeviceClient::onResultAvailable 向 API 发送捕获结果： 

1
2
3
4
5
6
7
8
9
10
// CameraDeviceClient.cpp
void CameraDeviceClient::onResultAvailable(const CaptureResult& result) {
    ...
    sp<hardware::camera2::ICameraDeviceCallbacks> remoteCb 
        = mRemoteCallback;
    if (remoteCb != NULL) {
        remoteCb->onResultReceived(result.mMetadata, 
            result.mResultExtras);
    }
}

而 API 中的回调是在 CameraDeviceImpl.java 中实现的： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
// CameraDeviceImpl.java
@Override
public void onResultReceived(CameraMetadataNative result,
        CaptureResultExtras resultExtras) throws RemoteException {
    int requestId = resultExtras.getRequestId();
    long frameNumber = resultExtras.getFrameNumber();
    ...
    synchronized(mInterfaceLock) {
      ...
      final CaptureCallbackHolder holder =
              CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);
      final CaptureRequest request = 
          holder.getRequest(resultExtras.getSubsequenceId());
      boolean isPartialResult = (resultExtras.getPartialResultCount() 
              < mTotalPartialCount);
      ...
      if (isPartialResult) {
        final CaptureResult resultAsCapture =
                new CaptureResult(result, request, resultExtras);
        // Partial result
        resultDispatch = new Runnable() {
        @Override
        public void run() {
          if (!CameraDeviceImpl.this.isClosed()) {
            if (holder.hasBatchedOutputs()) {
              for (int i = 0; i < holder.getRequestCount(); i++) {
                CameraMetadataNative resultLocal =
                        new CameraMetadataNative(resultCopy);
                CaptureResult resultInBatch = new CaptureResult(
                    resultLocal, holder.getRequest(i), resultExtras);

                holder.getCallback().onCaptureProgressed(
                    CameraDeviceImpl.this,
                    holder.getRequest(i),
                    resultInBatch);
              }
            } else {
              holder.getCallback().onCaptureProgressed(
                  CameraDeviceImpl.this,
                  request,
                  resultAsCapture);
            }
          }
        }
        };
        finalResult = resultAsCapture;
      } else {
        List<CaptureResult> partialResults =
                mFrameNumberTracker.popPartialResults(frameNumber);
        ...
        final TotalCaptureResult resultAsCapture = 
                new TotalCaptureResult(result, request, resultExtras,
                partialResults, holder.getSessionId());
        // Final capture result
        resultDispatch = new Runnable() {
        @Override
        public void run() {
          if (!CameraDeviceImpl.this.isClosed()){
            if (holder.hasBatchedOutputs()) {
              for (int i = 0; i < holder.getRequestCount(); i++) {
                ...
                TotalCaptureResult resultInBatch = new TotalCaptureResult(
                    resultLocal, holder.getRequest(i), resultExtras,
                    partialResults, holder.getSessionId());

                holder.getCallback().onCaptureCompleted(
                    CameraDeviceImpl.this,
                    holder.getRequest(i),
                    resultInBatch);
              }
            } else {
              holder.getCallback().onCaptureCompleted(
                  CameraDeviceImpl.this,
                  request,
                  resultAsCapture);
            }
          }
        }
        };
        finalResult = resultAsCapture;
      }
      ...
      // Fire onCaptureSequenceCompleted
      if (!isPartialResult) {
          checkAndFireSequenceComplete();
      }
    }
}

在 CameraDeviceImpl 中处理 CameraCaptureSession.CaptureCallback 各回调结果：如果返回的是部分结果，则回调 onCaptureProgressed ；如果返回最终结果，则回调 onCaptureCompleted 。整个预览、拍照、录像流程及回调分析完毕。

小结

  • Camera3Device::RequestThread
    这个类主要是处理预览、拍照、录像的请求 CaptureRequest 。
  • FrameProcessorBase.cpp
    这个类主要是处理请求后的回调函数，回调中会包含捕获的结果 CaptureResult 。
  • Condition
    不管是请求还是结果回调，因为是多线程处理，都涉及到条件变量的阻塞等待和通知机制。
  • CameraDeviceSession.CaptureCallback
    该回调接口都是在 CameraDeviceImpl 中实现的。

三者异同

预览、拍照、录像三者的流程基本一致，它们之间有如下异同：

  • 预览
    捕获请求模板为 CameraDevice.TEMPLATE_PREVIEW ；API 接口为 CameraCaptureSession.setRepeatingRequest ；repeating 值为 true 。
  • 拍照
    捕获请求模板为 CameraDevice.TEMPLATE_STILL_CAPTURE ；API 接口为 CameraCaptureSession.Capture ；repeating 值为 false 。
  • 录像
    捕获请求模板为 CameraDevice.TEMPLATE_RECORD ；API 接口为 CameraCaptureSession.setRepeatingRequest ；repeating 值为 true 。

也就是说，预览和录像仅仅是捕获请求模板不一样；而预览和拍照不管是模板，接口，repeating 值都不一样；但是它们三者最终在 Framework 中代码流程基本一致。

CameraServiceProxy 注册服务

AIDL

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
interface ICameraServiceProxy
{
    // CameraService 向代理服务发送消息，通知用户更新
    oneway void pingForUserUpdate();

    const int CAMERA_STATE_OPEN = 0;
    const int CAMERA_STATE_ACTIVE = 1;
    const int CAMERA_STATE_IDLE = 2;
    const int CAMERA_STATE_CLOSED = 3;
    const int CAMERA_FACING_BACK = 0;
    const int CAMERA_FACING_FRONT = 1;
    const int CAMERA_FACING_EXTERNAL = 2;

    // CameraService 向代理服务发送消息，通知相机设备状态更新
    oneway void notifyCameraState(String cameraId, int facing, 
            int newCameraState, String clientName);
}

从 AIDL 文件看出，CameraServiceProxy 主要是响应 CameraService 的请求，也就是向 Framework Java 发送消息。

流程图

CameraServiceProxy 服务名称：media.camera.proxy ；CameraServiceProxy 继承了 SystemService ，注册流程如下：

0109-android-camera-framework-CameraServiceProxy-addService.png

源码分析

先来看注册流程的源码，服务的标准注册流程： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
// CameraServiceProxy.java
public class CameraServiceProxy extends SystemService
        implements Handler.Callback, IBinder.DeathRecipient {
    ...
    private static final String CAMERA_SERVICE_BINDER_NAME 
        = "media.camera";
    public static final String CAMERA_SERVICE_PROXY_BINDER_NAME 
        = "media.camera.proxy";
    ...
    // 构造方法中，初始化线程相关
    public CameraServiceProxy(Context context) {
        super(context);
        mContext = context;
        mHandlerThread = new ServiceThread(TAG, 
            Process.THREAD_PRIORITY_DISPLAY, /*allowTo*/false);
        mHandlerThread.start();
        mHandler = new Handler(mHandlerThread.getLooper(), this);

        mNotifyNfc = SystemProperties.getInt(NFC_NOTIFICATION_PROP, 0)>0;
        if (DEBUG) Slog.v(...);
    }
    
    // onStart 主要是注册服务，并监听 User 相关广播
    @Override
    public void onStart() {
        mUserManager = UserManager.get(mContext);
        ...
        IntentFilter filter = new IntentFilter();
        filter.addAction(Intent.ACTION_USER_ADDED);
        filter.addAction(Intent.ACTION_USER_REMOVED);
        filter.addAction(Intent.ACTION_USER_INFO_CHANGED);
        filter.addAction(Intent.ACTION_MANAGED_PROFILE_ADDED);
        filter.addAction(Intent.ACTION_MANAGED_PROFILE_REMOVED);
        mContext.registerReceiver(mIntentReceiver, filter);

        // 注册 CameraServiceProxy 服务
        publishBinderService(CAMERA_SERVICE_PROXY_BINDER_NAME, 
            mCameraServiceProxy);
        publishLocalService(CameraServiceProxy.class, this);
        CameraStatsJobService.schedule(mContext);
    }
    ...
}

类中的 User 指的是 Android 多用户；再看回调接口的实现： 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// CameraServiceProxy.java
private final ICameraServiceProxy.Stub mCameraServiceProxy 
        = new ICameraServiceProxy.Stub() {
    @Override
    public void pingForUserUpdate() {
        notifySwitchWithRetries(30);
    }

    @Override
    public void notifyCameraState(String cameraId, int newCameraState,
            int facing, String clientName) {
        String state = cameraStateToString(newCameraState);
        String facingStr = cameraFacingToString(facing);
        if (DEBUG) Slog.v(...);
        updateActivityCount(cameraId, newCameraState,facing,clientName);
    }
};

小结

通常情况下 API 中，CameraManager 通过 ICameraService.aidl 向 CameraService 下发请求；而 CameraService 通过 ICameraServiceListener.aidl 发回回调。
而如果没有 API 请求的情况下，CameraService 无法向 Framework Java 发送信息，所以系统开机时注册了 CameraServiceProxy 服务，用于响应 CameraService 的回调。

其他

Camera 相关声音

1
2
3
4
5
6
7
8
9
10
11
12
13
14
void CameraService::loadSound() {
    ATRACE_CALL();

    Mutex::Autolock lock(mSoundLock);
    LOG1("CameraService::loadSound ref=%d", mSoundRef);
    if (mSoundRef++) return;

    mSoundPlayer[SOUND_SHUTTER] = newMediaPlayer(
        "/system/media/audio/ui/camera_click.ogg");
    mSoundPlayer[SOUND_RECORDING_START] = newMediaPlayer(
        "/system/media/audio/ui/VideoRecord.ogg");
    mSoundPlayer[SOUND_RECORDING_STOP] = newMediaPlayer(
        "/system/media/audio/ui/VideoStop.ogg");
}

CameraMetadata

CameraMetadataNative 和 CameraMetadata 是同一个类型，只是命名空间不一样。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
namespace android {

...
class CameraMetadata: public Parcelable {
...
}

namespace hardware {
namespace camera2 {
namespace impl {
using ::android::CameraMetadata;
typedef CameraMetadata CameraMetadataNative;
}
}
}

常见问题

API 2 + HAL 1

平台仅支持 HAL 1 时，API 2 在 openCamera 时，通过 CameraDeviceUserShim 将 API 2 转换为 API 1 ，即 HAL 1 + API 1 向下发起请求。
LegacyCameraDevice 会将 CAMERA API2 转换为 CAMERA API1 ，而 CameraDeviceUserShim 封装了 LegacyCameraDevice 。

AIDL 生成多类型文件

AIDL 可以同时生成 .java, .h, .cpp 文件，编译规则在 Android.bp 中配置。

总结

后续

  • 数据传递
    Binder 通信机制，数据传输限制在 1M ，那整个通信机制是如何传递图片的呢？以及预览的呢？传递的是什么？
    createCaptureSession 创建捕获会话时，配置输出流；当 setRepeatingRequest 发起预览请求时，回调结果为 CaptureResult ，它是如何和输出流关联的呢？
  • SurfaceFlinger 显示相关知识
  • Buffer 相关管理
    这些都是 API 1 模式下的数据流，不过有参考意义。

参考文档

0%