Is there a way to get the coordinates of the objects in an image?

[waatcher]

It's great that I am able to get the label for the objects in an image.

Is there a way to also get the coordinates of the identified objects?

[jared]

Hi @waatcher - right now we can only do this for human faces and not objects but it's definitely in our roadmap!

Face Model:
https://developer.clarifai.com/guide/publicmodels#face-detection

Celebrity Model:
https://developer.clarifai.com/guide/publicmodels#celebrity

[sabithpocker]

I am voting for this feature. This should have been there in the first place
I'm just a trespasser demanding for fundamental rights! LOL

Any estimate on when you will be releasing this feature and meanwhile any workarounds for us to implement this with any workarounds?

[jared]

No workarounds unfortunately but this is definitely on our roadmap!

We don't have a timetable on it yet but once we do I'll let the community know

[skol]

I was hoping Clarifai would gives the exact what coordinates of search for concept/object if it's found with high degree of certainty.

Looks like there's no image search API provider that can give you coordinates of the object. How is it even possible -- I mean if the object is detected, then surely bounding rectangle must be known, right?

Also, you should work on your documentation . Why it does NOT say anywhere what response object looks like?

 app.inputs.search(
    // this is the predicted concept
    {
      concept: {
        name: 'cat'
      }
    },
    // this is the user-supplied concept
    {
      concept: {
        type: 'input',
        name: 'dog'
      }
    }
  ).then(
    function(response) {
      // do something with response
    },
    function(err) {
      // there was an error
    }
  );

[jared]

Hi @skol - apologies for the docs here. We're currently working on some Swagger stuff which should be a lot more comprehensive and helpful.

Unfortunately we don't do object localization yet but it's definitely on our radar!

[skol]

@jared, thanks! How close are you to release object localisation feature?

[jared]

It's a long-term vision so it won't be out in the next few weeks, but hopefully soon enough. No immediate timeline yet

[jared]

Hi @skol - can I ask what your use case is with the localization?

[ame]

I was pretty sure I could acheive this goal by using the API called Prediction with Regions Feedback to train objects in the image. However, this thread seems to warn me it is not available yet. Then, my question is what is the reason for your have Prediction with Regions Feedback? What is the use case for the API?

clarifai.models.feedback(Clarifai.LOGO_MODEL, 'https://developer.clarifai.com/static/images/model-samples/logo-002.jpg', {
id: '{input_id}',
data: {
'regions': [
{
'region_info': {
'bounding_box': {
'top_row': 0.3,
'left_col': 0.2,
'bottom_row': 0.7,
'right_col': 0.8
},
'feedback': 'accurate'
},
'data': {
'concepts': [
{"id": "ai_TG4GHlnf", "value": true }
]
}
}
]
},
info: {
'eventType': 'annotation',
'outputId': '{output_id}',
'endUserId': '{end_user_id}',
'sessionId': '{session_id}'
}
})
.then(log)
.catch(log);

[jared]

Hi @ame! Right now the following models return coordinates and bounding boxes so that you know exactly where the objects are located:

Focus
Face
Celebrity
Demographic
Logo

For logo in particular, you may want to know exactly where a brand appears in an advertisement (and how many times). We don't offer this in custom models yet but it's definitely on our roadmap! Can you elaborate on your use case for it?

[ame]

Thanks for your update.

I'm trying to recognize a filled box in a time sheet with handwriting. I thought the handwritten boxes are similar with faces so I wanted to train the model same as face detection.

Definitely, this is very useful approach for many cases, I hope you guys working on this stuff to extend custom models.

[ArthurSav]

Custom models are not complete IMO if you don't return the coordinates of each detected concept.

Currently i'm trying to analyse an image constructed of concepts. For each concept detected i'd like to use region feedback to further analyse the region for colors or custom concepts that apply specifically to that region.

[jared]

Hi @ArthurSav - totally understand the use case and will let our product team know that it was requested again. It's on our roadmap but we don't have an immediate timetable quite yet