The Biggest Problem Yet

Originally when we devised this project, I thought the hardest problem would be to get people to use it and maybe to get an app onto iOS and Android phones. The phone problem has since disappeared and the “getting-people-to-use-it” problem is still to come but now there’s a new problem which will likely be the biggest: storage.

For a lost pet database, pictures play a crucial part. In fact, aside from geo-location, pictures are the most important as they are the best to represent a pet and help people recognize a lost one. Lost pet posters are usually bound to put just one picture of a pet up; we’re fine with three or four (though the owner would have to pick the best one to display at the fore-front).

Unfortunately, this would require a colossal amount of storage. Imagine that 50% of the pets lost in North America were reported on FindALostPet (ambitious, ’tis true, but if it works, then that’s the goal). For each pet, an average of two images is uploaded. This amount could perhaps be sustained by our servers at first but after a bit, there would be two many pictures to store.

Add in that people who see a pet might want to take a picture of it for the owner to confirm that it is, indeed, theirs. There would be way more of these pictures than the pictures uploaded by owners and if we thought we were swamped before, we’d have been sadly mistaken. Thousands of pictures a day wouldn’t be unexpected (or rather, I’d expect it to be more) and our hosting company would very quickly be either upping our prices or removing us from their system. Not good.

So how to solve this? Well, the simplest is to use a Content Distribution Network or CDN, but unfortunately those don’t come cheap. Since this is a .org projects with no particular plans of making money, it would be impossible to pay an extra $0.0001 per photo, however small that may seem.

Upon asking about a solution for my dilemma on a StackExchange forum, I was told that Imgur was a good solution. They seemed pretty nice as they had a good API and I was feeling hopeful again, until I read this in their terms of service:

Also, don’t use Imgur to host image libraries you link to from elsewhere, content for your website, advertising, avatars, or anything else that turns us into your content delivery network. If you do – and we will be the judge – or if you do anything illegal, in addition to any other legal rights we may have, we will ban you along with the site you’re hotlinking from, delete all your images, report you to the authorities if necessary, and prevent you from viewing any images hosted on Imgur.com. We mean it.

Since then, there haven’t been any other promising image CDNs. After all, a free product usually isn’t that and most of the .orgs I could find either didn’t have an API or were shutting down in just over a month.

In the end, we might have to get users to upload the images themselves to other websites such as Flickr, then link to them from FindALostPet. However, this would mean that the images would be completely out of our control and would make it more difficult for people who has just seen a lost pet to report it efficiently.

5 Comments


  1. I’d suggest you not worry too much about storage. Here are some considerations:

    * It’s extremely cheap. for example Digital Ocean[1] provides a server with 20GB of data for 5 bucks a month. That would hold at least a hundred thousand photos (assuming you limit/downsize the file which you would need to do anyway for reasonable performance).

    * By the time you have even thousands of photos, you will have plenty of users to ask for funding that should cover the cost or upgade fees.

    * Many already have photos of their pets online and even though they may be on imagur or Flickr I think it’s still worth it to let users hotlink the photos. For one major reason: it makes it easier for your users. don’t forget most users will probably get on your site when their pet is missing so they can’t take a photo.

    Also, the worst that can happen is for imagur to block your site, but really in your business model it becomes unimportant and at that point just prevent users from linking to imagur (besides I doubt they would care unless it stands out due to high traffic and by that time you’ll have users and can afford more storage).

    1. https://www.digitalocean.com/pricing/

    Cheers and Best of luck.

    Reply

    1. I know what you mean. Let me go through your points one by one.

      * It’s extremely cheap. Yup, that’s why we’re going with Google! Digital Ocean didn’t come across our radar so thanks for pointing it out!

      * By the time you have even thousands of photos, you will have plenty of users to ask for funding that should cover the cost or upgrade fees. This is true but we want to have it covered already. If we need to do future rounds of funding, we will put a notice on the homepage, similar to Wikipedia. Other than that, we want to build the system with to scale and image hosting is crucial to that.

      * Many already have photos of their pets online and even though they may be on imagur or Flickr I think it’s still worth it to let users hotlink the photos. I agree. We did run across this at one point but since all these websites would be beyond our control, they could track users, we wouldn’t be able to warn users about them connecting to all the websites and the permalinks might change. In the end, we may make it an option to use an image hosted on only particular hosts but we want to provide an option for people without an account on those.

      As to Imgur blocking us, it would be a discrepancy for the users and it might take us a while to find a different solution.

      Thanks for your thoughts! I’d love to talk more and I’m sure you have some other ideas.

      Cheers,
      Laef – The FindALostPet Team

      Reply

      1. Worth noting I just went to Imgur’s developer api site and found more info. Not sure, but I figure if they allow links from Facebook I’m pretty sure your app will be able to as well.

        http://api.imgur.com

        Commercial Usage
        Your application is commercial if you’re making any money with it (which includes in-app advertising), if plan on making any money with it, or if it belongs to a commercial organization.
        To use Imgur’s API commercially, you must first register your application. Once that’s done, you must register with Mashape. Mashape allows you to choose a pricing plan that fits your needs. From then on, the API endpoint is https://imgur-apiv3.p.mashape.com/ which must be used in replace of https://api.imgur.com/. Additionally, you must set a X-Mashape-Key request header with the key obtained from Mashape.

        Free Usage
        The Imgur API is free for non-commercial usage. Your application is probably free if you don’t plan on making any money with it, or if it’s open source.

        Heavy Free Usage
        If your application is free, but isn’t going to work with our rate limiting, then we might be willing to white-list it. Fill out this form submit your white-list request. We will review your request, and if granted, will give you all the access your awesome app desires!

        Cheers.
        Tim

        Reply

        1. Yes, those were the lines that got my hopes up. Then I read their terms of service, particularly the part about not turning them into your CDN and I decided it wasn’t worth trying. I sent them an email just to make sure but they never responded.

          Cheers,
          Laef

          Reply

Leave a Reply

Your email address will not be published. Required fields are marked *