OpenMAX IL Image+Video Common Configurations

A couple questions regarding the config structures in the IVCommon domain of the OpenMAX IL: (Section 4.2, page 211)

  1. Is there a standard order for the operations controlled by the IVCommon config structures (e.g. crop, rotate, scale, mirror, etc.)? Depending on what order these are performed in, the dimensions of buffers needed can be different.

  2. What is the typical behavior of the combination of a crop and output position? Here’s an example:
    input port resolution is 320 x 240
    crop rectangle: nLeft = 20, nTop = 30, nWidth = 300, nHeight = 200
    ouput position: nX = 0, nY = 0
    Should the output rectangle (cropped) be positioned at (0,0) or (20, 30)? If we wanted to put the output at (0,0), would we have to specify an output position of (-20, -30)?
    In other words, is the offset specified by the output position measured from the top-left corner of the port or the top-left corner of the crop rectangle?

Any insight would be greatly appreciated. Thanks!

Following are some comments for your questions:

  1. The specification only puts a minimum limit on a port’s buffer size (OMX_PARAM_PORTDEFINITIONTYPE.nBufferSize). So I think the caller should be responsible for ensuring that the buffers he is using are large enough for whatever operations he’s performing.

  2. I think all operations on a device should use the device’s coordinates and be independent of order. So the output position simply specifies the coordinates of the upper left corner of the output and crop specifies the visible portion of the output.

So yes the output rectangle will be positioned at (20, 30).

And to have the output at (0, 0) will require shifting the crop rectangle to (0, 0) and the output position will simply control what part of the image gets displayed in that region.

With these assumptions, the output position will be measured from the top-left corner of the port.

Thanks for your input.

Regarding question 1, I was referring to internal buffers allocated by the component during the Loaded->Idle transition. Since the component has no idea what scale factors to expect, I guess it would have to just allocate at least the bare minimum and return an error on OMX_SetConfig if the client tries to set a value that would require more.

Another question along the same lines is what should a component do if the client sets a configuration that would put part of an image outside the frame? If the component clamps the values, should it return the clamped values or the originals on a subsequent OMX_GetConfig call?


One more thing…

What is the difference between InputCrop and OutputCrop? For a component with input and output ports is it necessary to support both of them on all ports, or can you just have InputCrop on the inputs and OutputCrop on the outputs?


Regarding question 1, a component can implicitly allocate its own buffers but this is not mandatory. This was one of the few problems I expect with implicit buffers so I prefer to avoid them. Another problem is that when a component transitions LOADED->IDLE, it must call UseBuffer on tunneled ports to share its internal buffer with the tunneled component. But if the tunneled component is not in the IDLE state yet, then the UseBuffer call will fail - so this makes the order of “set state to IDLE” commands important. If the client will always make the allocate or use buffer calls, then it would be easier for him to ensure that all components are in the IDLE state before the buffers are assigned.

Generally its a good practice to keep the values of get and set calls consistent. So the decision here will be whether to fail such set value calls or to work around them internally.

The InputCrop and OutputCrop operations are not documented so we can only assume what they mean. I think these are from a port’s perspective; InputCrop crops the image before a port’s processing and OutputCrop crops a port’s processed output (regardless of whether the port is input or output). I don’t think either of these are mandatory for a component. You can implement them if it makes sense for your component.