Subject Crop and RLE Mask
Crop around the detected subject and decode the returned subject mask from result.json.
Use AutoMode: "Subject" when the crop should be based on the detected foreground object instead of pose landmarks or manual coordinates. This is useful for product photos, still-life images, catalog automation, and any workflow where the subject is not necessarily a person.
Two Subject Crop Workflows
Subject Crop has two common API paths. Use the first when your application only needs the final cropped image. Use the second when you also need the detected subject mask for custom UI, compositing, QA, or downstream automation.
1. Return a Ready Image
Set Layer: 0 and an image outputFormat, such as jpeg. The API returns the final cropped file from /retoucher/getFile/{id}.
2. Return JSON with Mask
Set Layer: 1 with outputFormat: "zip". The ZIP contains result.json with subjectMask RLE metadata, which you can decode and draw yourself.
The square example below crops the detected object into a fixed 1 : 1 frame with a small tolerance around the mask. For natural product framing, use AspectRatio: "Free" as shown in the fruit gallery.
subjectMask drawn as an overlay. Download overlay PNGMetadata archive
Free Aspect Ratio Fruit Crops
Use AspectRatio: "Free" when the output should follow the detected subject shape instead of forcing a square or catalog ratio. This is useful for mixed product batches: a banana can stay wide, while a pear or apple can use a taller crop. The examples below are ready Layer: 0 Cloud API outputs generated from subject detection, not raw mask visualizations.
Show Free aspect ratio subject crop JSON
{
"mode": "professional",
"outputFormat": "jpeg",
"tasks": [
{
"Plugin": "Crop",
"Layer": 0,
"User Params": {
"AutoMode": "Subject",
"AutoStraightenMode": "Manual",
"AspectRatio": "Free",
"OnlyMainSubject": true,
"CustomMargins": true,
"CustomMarginX": 0.5,
"CustomMarginY": 0.5,
"Extend": 12,
"AllowOutOfBounds": false
}
}
]
}Margins and Extension
Extend adds extra space around the detected subject. It is a percentage value from 0 to 100. CustomMarginX and CustomMarginY control how that extra space is distributed. Use 0.5 / 0.5 to keep the subject centered. Values below or above 0.5 bias the extra space toward one side, which is useful when the subject should sit higher, lower, left, or right in the frame. If Extend is 0, custom margins do not change the crop because there is no extra margin to distribute.
X: 0.5, Y: 0.5. Extra space is balanced.
X below or above 0.5 puts more added space on one side.
Y below or above 0.5 puts more added space above or below.
Extend: 0. There is no added space for margins to distribute.
| Goal | Settings | Result |
|---|---|---|
| Centered subject | Extend: 12, CustomMarginX: 0.5, CustomMarginY: 0.5 | Equal extra space around the detected subject. |
| Horizontal composition shift | Extend: 12, CustomMarginX: 0.35 or 0.65, CustomMarginY: 0.5 | More of the added margin goes to one horizontal side. |
| Vertical composition shift | Extend: 12, CustomMarginX: 0.5, CustomMarginY: 0.35 or 0.65 | More of the added margin goes above or below the subject. |
| No extra space | Extend: 0, any CustomMarginX / CustomMarginY | Custom margin values have no visible effect. |
Subject Crop Request
Use this request when you want the API to return a finished image. The input image is uploaded once, Crop finds the main foreground object, builds the crop frame using the requested AspectRatio, and /retoucher/getFile/{id} returns a JPEG crop.
In the pear example above, AspectRatio: "1 : 1" produces a square crop JPEG. Use another fixed ratio or AspectRatio: "Free" when the output should not be square. No mask decoding is needed on the client side.
| Field | Meaning in this example |
|---|---|
outputFormat: "jpeg" | Return a normal image file, not a ZIP archive. |
Layer: 0 | Flatten the Crop result into the output image. |
AutoMode: "Subject" | Use the detected subject mask to choose the crop frame. |
AspectRatio: "1 : 1" | Use a square product/catalog crop for this example. Other fixed ratios and "Free" are also supported. |
Extend: 2 | Add a small amount of space around the detected object. Extend is specified in percent and accepts values from 0 to 100. |
| Output | A ready JPEG like Crop_Subject_Result.jpg. |
Show square subject crop JSON
{
"mode": "professional",
"outputFormat": "jpeg",
"tasks": [
{
"Plugin": "Crop",
"Layer": 0,
"User Params": {
"AutoMode": "Subject",
"AutoStraightenMode": "Manual",
"AspectRatio": "1 : 1",
"OnlyMainSubject": true,
"CustomMargins": true,
"CustomMarginX": 0.5,
"CustomMarginY": 0.5,
"Extend": 2,
"AllowOutOfBounds": false
}
}
]
}Subject Metadata Request
Use ZIP output or Layer: 1 when the client needs subjectMask metadata instead of only the flattened image.
Use this request when your application wants to draw, inspect, or post-process the mask. The API returns a ZIP file. Inside it, result.json contains the same Crop prediction metadata plus subjectMask, encoded as RLE.
This path does not return the final crop as a plain JPEG. Your client downloads the ZIP, reads result.json, decodes subjectMask.counts, and can draw the mask overlay like the blue pear example above.
| Field | Meaning in this example |
|---|---|
outputFormat: "zip" | Return a ZIP archive containing result.json. |
Layer: 1 | Ask Crop to write metadata instead of only flattening the image result. |
subjectMask | Binary mask of the detected subject. It is stored in result.json, not in the request JSON. |
encoding: "rle-v1" | The mask pixels are compressed as run lengths in row-major order. |
width / height | Mask dimensions. After decoding, the number of pixels must equal width * height. |
| Output | A ZIP like Crop_Subject_Metadata.zip, containing result.json. The decoded mask can be drawn as this overlay PNG. |
Show subject metadata JSON
{
"mode": "professional",
"outputFormat": "zip",
"tasks": [
{
"Plugin": "Crop",
"Layer": 1,
"User Params": {
"AutoMode": "Subject",
"AutoStraightenMode": "Manual",
"AspectRatio": "1 : 1",
"OnlyMainSubject": true,
"CustomMargins": true,
"CustomMarginX": 0.5,
"CustomMarginY": 0.5,
"Extend": 2,
"AllowOutOfBounds": false
}
}
]
}RLE Format
| Field | Description |
|---|---|
encoding | Currently rle-v1. |
width / height | Mask size in pixels. The mask is stored row-major. |
startsWith | 0 means the first run is background. 1 means the first run is subject. |
counts | Alternating run lengths. After each count, the value flips between background and subject. |
largestMaskIndex | Connected-component metadata for the largest detected subject. The RLE itself is binary. |
Python RLE Decoder and Overlay
The runnable source file is also available as examples/decode_subject_mask.py.
import json
from pathlib import Path
from PIL import Image
DOCS_DIR = Path(__file__).resolve().parents[1]
SOURCE_PATH = DOCS_DIR / "CropDocs-Items_DSC_2416.jpg"
JSON_PATH = DOCS_DIR / "crop_subject_metadata" / "result.json"
OUTPUT_PATH = DOCS_DIR / "Crop_Subject_Mask_Overlay.png"
def decode_subject_mask_rle(mask_json):
width = int(mask_json["width"])
height = int(mask_json["height"])
value = 1 if mask_json.get("startsWith") else 0
pixels = []
for run in mask_json["counts"]:
pixels.extend([value] * int(run))
value = 0 if value else 1
expected = width * height
if len(pixels) != expected:
raise ValueError(f"Decoded {len(pixels)} pixels, expected {expected}")
image = Image.new("L", (width, height))
image.putdata(pixels)
return image
data = json.loads(JSON_PATH.read_text(encoding="utf-8"))
mask_json = data["plugins"][0]["subjectMask"]
mask = decode_subject_mask_rle(mask_json)
source = Image.open(SOURCE_PATH).convert("RGBA")
mask = mask.resize(source.size, Image.Resampling.NEAREST)
overlay = Image.new("RGBA", source.size, (0, 128, 255, 0))
overlay.putalpha(mask.point(lambda p: 112 if p else 0))
Image.alpha_composite(source, overlay).save(OUTPUT_PATH)JavaScript RLE Decoder and Canvas Overlay
The runnable source file is also available as examples/decode_subject_mask.js.
function decodeSubjectMaskRle(maskJson) {
const width = Number(maskJson.width);
const height = Number(maskJson.height);
const output = new Uint8Array(width * height);
let value = maskJson.startsWith ? 1 : 0;
let offset = 0;
for (const run of maskJson.counts) {
output.fill(value, offset, offset + Number(run));
offset += Number(run);
value = value ? 0 : 1;
}
if (offset !== output.length) {
throw new Error(`Decoded ${offset} pixels, expected ${output.length}`);
}
return { width, height, pixels: output };
}
function drawSubjectMaskOnCanvas(ctx, image, subjectMask) {
const decoded = decodeSubjectMaskRle(subjectMask);
ctx.drawImage(image, 0, 0);
const maskCanvas = new OffscreenCanvas(decoded.width, decoded.height);
const maskCtx = maskCanvas.getContext('2d');
const imageData = maskCtx.createImageData(decoded.width, decoded.height);
for (let i = 0; i < decoded.pixels.length; i += 1) {
const a = decoded.pixels[i] ? 112 : 0;
imageData.data[i * 4 + 0] = 0;
imageData.data[i * 4 + 1] = 128;
imageData.data[i * 4 + 2] = 255;
imageData.data[i * 4 + 3] = a;
}
maskCtx.putImageData(imageData, 0, 0);
ctx.drawImage(maskCanvas, 0, 0, image.width, image.height);
}Full result.json Example
The full example below includes the complete subjectMask.counts array.
Show full result.json
{
"jsonVersion": "1",
"plugins": [
{
"AI predicted": {
"Angle": -0.0,
"RotInvariant": 0.7620668411254883
},
"operationIndex": 0,
"pluginName": "Crop",
"poseLandmarks": [
[
0.17434480563551188,
-0.027062764493308387
],
[
0.21151109033077958,
-0.07440895257994186
],
[
0.1369331278251484,
-0.06882292939949465
],
[
0.24647846671566367,
-0.06870380667752998
],
[
0.09429104230133814,
-0.049836504307351245
],
[
0.476942777633667,
-0.04130150377750397
],
[
-0.026286736130714417,
0.37871527671813965
],
[
0.9026992321014404,
0.48666006326675415
],
[
0.0593884140253067,
1.0057748556137085
],
[
0.8031997680664062,
0.5564185976982117
],
[
-0.19922615587711334,
0.8251978158950806
],
[
1.1207892894744873,
1.0876448154449463
],
[
0.7707972526550293,
1.328694224357605
],
[
1.158691644668579,
1.0754677057266235
],
[
0.773831307888031,
1.1648155450820923
],
[
1.0618743896484375,
1.3289319276809692
],
[
1.0261671543121338,
1.2343072891235352
],
[
0.1738161135558039,
0.0057850259636366
],
[
0.20215803719311953,
0.00957647947467366
],
[
0.14534283507708462,
0.013093204312278013
],
[
0.17269521286385137,
0.06975993972386262
],
[
0.7546296715736389,
0.6166388392448425
],
[
0.7556363940238953,
0.5241447687149048
],
[
-0.40994536876678467,
0.6785233020782471
],
[
-0.4075469970703125,
0.7177042961120605
],
[
1.030322790145874,
1.3849692344665527
],
[
1.0381921529769897,
1.3141342401504517
],
[
1.1987611055374146,
1.1682474613189697
],
[
1.0189801454544067,
1.2213608026504517
]
],
"posePrediction": {
"bbox": {
"bottom": 0.989516019821167,
"format": "normalized_ltrb",
"left": 0.014289194718003273,
"right": 0.9776310920715332,
"top": 0.006422898732125759
},
"classLabels": [
"noPerson",
"singlePerson",
"multiplePeople"
],
"classLogits": [
4.456557273864746,
-3.867187023162842,
-5.338537216186523
],
"classProbabilities": [
0.9997016787528992,
0.0002426131977699697,
5.5707663705106825e-05
],
"keypointConfidences": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"keypointCount": 29,
"modelOutputLayout": "keypoints58_bbox4_class3_conf29",
"multiplePeopleProbability": 5.5707663705106825e-05,
"noPersonProbability": 0.9997016787528992,
"singlePersonProbability": 0.0002426131977699697
},
"subjectMask": {
"counts": [
5512,
3,
253,
3,
253,
4,
252,
4,
251,
5,
251,
5,
251,
5,
251,
4,
251,
5,
251,
5,
251,
5,
251,
5,
250,
6,
246,
10,
243,
13,
241,
15,
239,
22,
233,
25,
230,
28,
228,
29,
226,
31,
224,
33,
222,
35,
220,
36,
219,
38,
218,
39,
216,
40,
216,
41,
214,
43,
212,
44,
212,
44,
211,
46,
210,
46,
209,
47,
208,
49,
206,
50,
205,
52,
203,
53,
202,
55,
200,
57,
198,
59,
196,
61,
194,
63,
192,
64,
191,
67,
189,
67,
188,
69,
187,
70,
185,
71,
184,
73,
182,
75,
181,
75,
180,
77,
179,
77,
178,
79,
177,
79,
177,
80,
176,
80,
175,
82,
174,
82,
174,
82,
174,
82,
173,
83,
173,
83,
173,
84,
172,
84,
172,
84,
172,
84,
172,
84,
172,
84,
172,
84,
172,
84,
172,
83,
173,
83,
173,
83,
173,
83,
174,
82,
174,
82,
174,
81,
176,
80,
176,
79,
177,
79,
178,
78,
178,
77,
180,
76,
180,
75,
182,
74,
182,
74,
183,
72,
184,
71,
186,
69,
188,
68,
189,
66,
190,
65,
193,
62,
194,
61,
197,
57,
200,
56,
201,
53,
204,
52,
207,
47,
210,
45,
213,
41,
216,
39,
219,
35,
223,
32,
228,
26,
232,
22,
239,
13,
247,
6,
10357
],
"encoding": "rle-v1",
"height": 171,
"largestMaskIndex": 2,
"startsWith": 0,
"width": 256
}
}
]
}