Recommendation Image for Custom Training & Latency Time

[Charles808]

Hi !

Thank you for the supportive response until now.

I have a couple question :
* Do you have any recommendation about what we have to pay attention to, when choosing image for custom training ?
Such as, is it better to have multiple target objects inside one image or It's better if there is only one target object inside the image ? What about the background ? Plain background is better than mixed background ? Size, orientation, etc ?
If you have any suggestions about this, I would like to hear about it.

• Second question.
  I develop Clarifai on Android platform.
  What I want to know. What is the expected latency time from uploading the image until I got the predictions result ?
  In my case, It's variating from 2-10 secs, depending on image size. Is it normal ?
  I just wondering, If I always get more than 3 secs latency, I think It will be impossible to implement it in video stream with live prediction. Not impossible actually, but the performance will be disappointing. Any comment about this ?

Thank you

Warm regards.

[jared]

Hi Charles! Regarding your questions:

1) It is best to have a mix here actually so that our system can see the object in a variety of scenarios. The more images, angles and zooms that you have, the better, and you'll definitely want to train a concept by itself (e.g. with nothing else present) in at least 10+ images.

The background shouldn't make much of a difference for this as our algorithm analyzes items at the pixel level. The size shouldn't matter much either, although 512x512px is our generally recommended format.

2) Your custom model should return results in less than a second, so if you're seeing 2-10 seconds then it is either due to: a) Image size and/or b) Internet connection. If you're using large PNGs I would recommend scaling down to JPEGs instead since we need to download any images and that directly affects latency.

Hope this helps!

[Charles808]

Understood ! Thanks a lot Jared !

[Charles808]

Hi Jared !

I have further questions about this matter.

So, I have tried yours and Liz advices both from e-mail and from the community.

Here are what I have tried so far :
1. I stop to crop the image only to the target object, since Liz said It wont help.
2. And then I tried to experiment with several positives and negative images combination
- I have two concepts for now.
I tried with 20 positives images and 20 negatives images each concepts.
But, the negatives images is actually from the other concepts positive images.
For example, Concept A have 20 negatives images whihch come from Concept B, 20 positive images.
And the same for the Concept B negative images.
I got the best result with this. Using one test image, I got around 80% in prediction result.
- So, I tried to add more images to 30 for positive only for both concepts
The result of accuracy Concept A become 70% and Concept B close to 90%
- Tried add more negatives images. So, now I have 30 pos & 30 neg images for each concepts.
Both Concepts accuracy decrease a little.
- Tried with 30 pos images and 20 neg images.
But now the negative images is the same for both concepts.
Also, the negative images is no longer from positives images of the other concepts, but it's "view" images.
The result of accuracy Concept A 60%, Concept B 70%
- I also tried to mix negative samples with view images and positive images of the other concepts, but still not
a satisfying result.
3. My model name is drink_can. Can you see what I have done wrongly ? Because you could see that from my trials, more images actually makes the accuracy percentages lower.
4. How many images do Clarifai need to make a model with accuracy reaching more than 90% ?
5. Any recommended ratio between positives and negatives images ?
6. For the latency. My android phone is already using LTE connection, does 2-10 secs is still normal ?
The picture itself are usually taken from camera.

Thank you

Warm regards.

[jared]

Hi Charles!

This isn't an exact science unfortunately, but I can tell you that you'll want to add more Positives than Negatives to your concepts. Adding the latter will help to a certain extent, but I would recommend having at least a 3:1 or 4:1 ratio of P to N here.

To obtain a 99.99% probability you'll either need to analyze the exact same picture that was used for training or train with a very large amount of data. You're more likely to see numbers in the 90+ range for a well-trained model, which is usually good enough to tell if something appears or not.

The latency numbers are a bit concerning though. Where are you uploading these items from?

[Charles808]

Hi Jared !

Thank you for giving me some important clues.

You mean where I am located ?
So, here are some informations which might be useful for you.

I uploaded the image from Android Smartphones.
I tried in two places in Asia, which are in Indonesia using LG G3 and in South Korea using Samsung Galaxy S7.
Both are using the same app.
The app itself was developed based on the clarifai starter. Especially, in AsyncTask when uploading the images.
I also used the ClarifaiUtil.java which has compressing function for the image before being uploaded.
The images itself could come from both Gallery or Camera.
And in both cases/places are having the same latency, using 4G internet connection.

Any ideas how I could improve the response speed ?

Oh, one more thing...
Sometimes I got "Error while contacting Clarifai API" message.
Which means the response in unsuccessful.

I have plan to use it for live stream.
But with this kind of latency... I don't think It will work.

Warm regards.

[jared]

Unfortunately this looks like a location issue. We've seen other customers in Asia experience the same kind of latencies and we're working on a good solution for it.

The good news is that there are workarounds for this. Since we are hosted in the Eastern U.S., you could potentially move your files to a server here (like AWS S3), or you could use our Mobile SDK solution which is currently in Alpha. The latter will allow you to do everything locally on your phone and should be in Beta soon enough!

[Charles808]

Hi Jared !

Would you mind explained more in how to do it, for both solutions ?

What files that I need to move ? Do you mean the positive and negative images ?
Until now I'm using Preview UI and inserting the images for training.
How can I moved it to a server like AWS 3 ?

And, do you think I could get the Mobile SDK Alpha Version for testing ?
Or should I wait for the Beta version for it to be available for developers ?

Warm regards.

[jared]

Is it the upload the UI that takes a long time or the predictions on external URLs? Just want to make sure I understand the problem here.

I'll also add you to the alpha list!

[Charles808]

Okay.

Please let me know if I already added to the alpha list, so I could try it directly as soon as possible.

Regarding to your question :
- I don't think I have problem with the latency time, when uploading images to Preview UI.
- The latency was long starting from when I upload the image to be predicted from the Android app, until I got the prediction result back. Like I said before the upload function part is following your clarifai-starter example.
Honestly, I don't know which one is taking longer, the upload part or when I request for the prediction result. I wonder about it myself. If you could give me some pointer that would be awesome.

Warm regards.