unified_apple_vision 🍎

A plugin for using Apple Vision Framework with Flutter, designed to integrate multiple APIs into one plugin and process multiple analysis requests at once.

Features ⚙️ & Requirements 🧩

Minimum Requirements: iOS: 13.0+, macOS: 15.0+

Warning

[*] Note that this differs from the VisionAPI minimum requirements.

Status: ✅ Complete ⚠️ Problematic 👨‍💻 In Progress ❌ Not Yet

Vision API	Request	Status	iOS	macOS	description
Still Image Analysis	Classify Image	✅	13.0+	10.15+	A request to classify an image.
	Generate Image Feature Print	✅	13.0+	10.15+	An image-based request to generate feature prints from an image. (FeaturePrint — vector image descriptor similar to a word vector)
Image Sequence Analysis	Generate Person Segmentation	❌	15.0+	12.0+	Produces a matte image for a person it finds in the input image.
	Generate Person Instance Mask	❌	17.0+	14.0+	Produces a mask of individual people it finds in the input image.
	Detect Document Segmentation	❌	15.0+	12.0+	Detects rectangular regions that contain text in the input image.
Saliency Analysis	Generate Attention Based Saliency Image	❌	13.0+	10.15+	Produces a heat map that identifies the parts of an image most likely to draw attention.
	Generate Objectness Based Saliency Image	❌	13.0+	10.15+	Generates a heat map that identifies the parts of an image most likely to represent objects.
Object Tracking	Track Rectangle	⚠️	*11.0+	*10.13+	Tracks movement of a previously identified rectangular object across multiple images or video frames.
	Track Object	⚠️	*11.0+	*10.13+	Tracks the movement of a previously identified object across multiple images or video frames.
Rectangle Detection	Detect Rectangle	✅	*11.0+	*10.13+	Finds projected rectangular regions in an image.
Face and Body Detection	Detect Face Capture Quality	✅	13.0+	10.15+	Produces a floating-point number that represents the capture quality of a face in a photo.
	Detect Face Landmarks	✅	*11.0+	*10.13+	Finds facial features like eyes and mouth in an image.
	Detect Face Rectangles	✅	*11.0+	*10.13+	Finds faces within an image.
	Detect Human Rectangles	✅	13.0+	10.15+	Finds rectangular regions that contain people in an image.
Body and Hand Pose Detection	Detect Human Body Pose	❌	14.0+	11.0+	Detects a human body pose.
	Detect Human Hand Pose	❌	14.0+	11.0+	Detects a human hand pose.
3D Body Pose Detection	Detect Human Body Pose 3D	❌	17.0+	14.0+	Detects points on human bodies in three-dimensional space, relative to the camera.
Animal Detection	Recognize Animals	✅	13.0+	10.15+	Recognizes animals in an image. (Now only dogs and cats are supported.)
Animal Body Pose Detection	Detect Animal Body Pose	❌	17.0+	14.0+	Detects an animal body pose.
Trajectory Detection	Detect Trajectories	❌	14.0+	11.0+	Detects the trajectories of shapes moving along a parabolic path.
Contour Detection	Detect Contours	❌	14.0+	11.0+	Detects the contours of the edges of an image.
Optical Flow	Generate Optical Flow	❌	14.0+	11.0+	Generates directional change vectors for each pixel in the targeted image.
	Track Optical Flow	❌	17.0+	14.0+	Determines the direction change of vectors for each pixel from a previous to current image.
Barcode Detection	Detect Barcodes	✅	*11.0+	*10.13+	Detects barcodes in an image.
Text Detection	Detect Text Rectangles	✅	*11.0+	*10.13+	Finds regions of visible text in an image.
Text Recognition	Recognize Text	✅	13.0+	10.15+	Finds and recognizes text in an image.
Horizon Detection	Detect Horizon	❌	*11.0+	*10.13+	Determines the horizon angle in an image.
Image Alignment	Translational Image Registration	❌	*11.0+	*10.13+	Determines the affine transform necessary to align the content of two images.
	Track Translational Image Registration	❌	17.0+	14.0+	An image analysis request, as a stateful request you track over time, that determines the affine transform necessary to align the content of two images.
	Homographic Image Registration	❌	*11.0+	*10.13+	Determines the perspective warp matrix necessary to align the content of two images.
	Track Homographic Image Registration	❌	17.0+	14.0+	An image analysis request, as a stateful request you track over time, that determines the perspective warp matrix necessary to align the content of two images.
Image Background Removal	Generate Foreground Instance Mask	❌	17.0+	14.0+	Generates an instance mask of noticable objects to separate from the background.

Important

If you have a request, please make an issue.

Install 📦

Add this to your pubspec.yaml:

unified_apple_vision: ^latest

Usage 🕹

// initialize
final vision = UnifiedAppleVision();

// create input image
final input = VisionInputImage(
  bytes: image.bytes,
  size: image.size,
);

// analyze
vision.analyze(
  image: input,
  requests: [
    // add requests you wish to perform
    VisionRecognizeTextRequest(
      onResults: (results) {
        final observations = results.ofRecognizeTextRequest; // get casted results
        // some action
      },
      onError: (error) {
        // handle error
      },
    ),
  ],
);

Documents 📘

Contributing 🤝

Contributions are welcome! Please feel free to submit a Pull Request.

Adding New Request or Observation

If you want to add a new request or observation class, you can use the tools.py script to generate the boilerplate code for you.

python3 tools.py request --name VNDetectRectanglesRequest --ios 11.0 --macos 10.13
python3 tools.py observation --name VNRectangleObservation --extends VNDetectedObjectObservation

License 📜

This project is licensed under the MIT License - see the LICENSE file for details.