前言
- 最近在研究苹果的core ml模型，应该说模型本身已经封装的非常完善，训练模型，提供的方法接口都是现成的
- 这篇博客主要是从头到尾解读.mlmodel生成的方法头文件
- 其实认真钻读一篇代码文件是一件很有趣的事，在研读这篇代码工程中，读了很多文章，扩展了很多知识。最近一直在忙实验室纳新的事，好久没认真学习了，今天过的很充实快乐
- 建议以后还是多看apple官方文档，由谷歌翻译的情况下，并没有什么看不懂的
参考文章
如何打开方法文件
- 点开.mlmodel文件，在Model Class中点击右侧的小箭头
- 该文件自动生成，且和苹果其他源文件一样，只能看到.h文件，无法修改内部实现
- 我用的是自己训练的一个CatorDog.mlmodel
详细分块的代码解读
//第一部分：CatorDogInput
/// Input image to be classified as color (kCVPixelFormatType_32BGRA) image buffer, 299 pixels wide by 299 pixels high
@property (readwrite, nonatomic) CVPixelBufferRef image;
- (instancetype)init NS_UNAVAILABLE;
- (instancetype)initWithImage:(CVPixelBufferRef)image;
@end
1. CVPixelBufferRef：是一种像素图片类型，由于CV开头，所以它是属于 CoreVideo 模块的。
2. RGB32图像每个像素用32比特位表示，占4个字节，R，G，B分量分别用8个bit表示，存储顺序为B，G，R，最后8个字节保留。注意：在内存中RGB各分量的排列顺序为：BGRA BGRA BGRA ......。
3. 输入图片以299像素宽高
//第二部分：CatorDogOutput
/// Probability of each category as dictionary of strings to doubles
@property (readwrite, nonatomic, strong) NSDictionary<NSString *, NSNumber *> * classLabelProbs;
/// Most likely image category as string value
@property (readwrite, nonatomic, strong) NSString * classLabel;
- (instancetype)init NS_UNAVAILABLE;
- (instancetype)initWithClassLabelProbs:(NSDictionary<NSString *, NSNumber *> *)classLabelProbs classLabel:(NSString *)classLabel;
@end
1. classLabelProbs example {
Cat = "0.9999999695704587";
Dog = "3.042954133459557e-08";
} 该字典会由一个NSString（显示识别出的所有可能物品）以及一个NSNumber（显示该物品的可能性）
2. classLabel example Cat
该NSString显示最有可能的物品
//第三部分：CatorDog
@property (readonly, nonatomic, nullable) MLModel * model;
- (nullable instancetype)init;
- (nullable instancetype)initWithContentsOfURL:(NSURL *)url error:(NSError * _Nullable * _Nullable)error;
- (nullable instancetype)initWithConfiguration:(MLModelConfiguration *)configuration error:(NSError * _Nullable * _Nullable)error API_AVAILABLE(macos(10.14), ios(12.0), watchos(5.0), tvos(12.0)) __attribute__((visibility("hidden")));
- (nullable instancetype)initWithContentsOfURL:(NSURL *)url configuration:(MLModelConfiguration *)configuration error:(NSError * _Nullable * _Nullable)error API_AVAILABLE(macos(10.14), ios(12.0), watchos(5.0), tvos(12.0)) __attribute__((visibility("hidden")));
/**
Make a prediction using the standard interface
@param input an instance of CatorDogInput to predict from
@param error If an error occurs, upon return contains an NSError object that describes the problem. If you are not interested in possible errors, pass in NULL.
@return the prediction as CatorDogOutput
*/
- (nullable CatorDogOutput *)predictionFromFeatures:(CatorDogInput *)input error:(NSError * _Nullable * _Nullable)error;
/**
Make a prediction using the standard interface
@param input an instance of CatorDogInput to predict from
@param options prediction options
@param error If an error occurs, upon return contains an NSError object that describes the problem. If you are not interested in possible errors, pass in NULL.
@return the prediction as CatorDogOutput
*/
- (nullable CatorDogOutput *)predictionFromFeatures:(CatorDogInput *)input options:(MLPredictionOptions *)options error:(NSError * _Nullable * _Nullable)error;
/**
Make a prediction using the convenience interface
@param image Input image to be classified as color (kCVPixelFormatType_32BGRA) image buffer, 299 pixels wide by 299 pixels high:
@param error If an error occurs, upon return contains an NSError object that describes the problem. If you are not interested in possible errors, pass in NULL.
@return the prediction as CatorDogOutput
*/
- (nullable CatorDogOutput *)predictionFromImage:(CVPixelBufferRef)image error:(NSError * _Nullable * _Nullable)error;
/**
Batch prediction
@param inputArray array of CatorDogInput instances to obtain predictions from
@param options prediction options
@param error If an error occurs, upon return contains an NSError object that describes the problem. If you are not interested in possible errors, pass in NULL.
@return the predictions as NSArray<CatorDogOutput *>
*/
- (nullable NSArray<CatorDogOutput *> *)predictionsFromInputs:(NSArray<CatorDogInput*> *)inputArray options:(MLPredictionOptions *)options error:(NSError * _Nullable * _Nullable)error API_AVAILABLE(macos(10.14), ios(12.0), watchos(5.0), tvos(12.0)) __attribute__((visibility("hidden")));
@end
1. - (nullable instancetype)initWithContentsOfURL:(NSURL *)url error:(NSError * _Nullable * _Nullable)error;
一般使用该方法初始化我们的mlmodel，其中url就是我们存放.mlmodel的地方，使用NSBundle获取
example：NSURL * modelUrl = [[NSBundle mainBundle] URLForResource:@"CatorDog" withExtension:@"mlmodelc"];
放一段apple官方文档的解释：
name
The name of the resource file.
If you specify nil, the method returns the first resource file it finds with the specified extension.
extension
The extension of the resource file.
If extension is an empty string or nil, the extension is assumed not to exist and the file URL is the first file encountered that exactly matches name.
如果extension是空字符串或nil，则假定扩展名不存在，文件URL是遇到的第一个与name完全匹配的文件。
就是说extension填写其扩展名,不填的话就会直接找第一个名字匹配的文件
2. - (nullable CatorDogOutput *)predictionFromImage:(CVPixelBufferRef)image error:(NSError * _Nullable * _Nullable)error;使用该方法来创建Output，其中image的建立，使用CVPixelBufferCreate创建
example ：
CVPixelBufferRef imgBuf = NULL;
CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault,
size.width,
size.height,
kCVPixelFormatType_32ARGB,
(__bridge CFDictionaryRef) @{(__bridge NSString *) kCVPixelBufferIOSurfacePropertiesKey: @{}},
&imgBuf);
