book collections email follower instructable user
Picture of FindyBot3000 - a Voice Controlled Organizer

As any maker can attest, over the years, one collects a lot of parts. Perhaps it all started with a few resistors and LEDs, a breadboard, and an Arduino. But soon there were motors and motor drivers, IR rangefinders and tactile switches. Accelerometers, microphones, heatshrink tubes, potentiometers, relays, hookup wire, all those things salvaged from that printer you took apart... and the list goes on.

Now, organizing such a collection of goodies is no easy task. Some may have turned to baggies, for others, boxes. And then there were those who filled shelves with boxes, filled with baggies, filled with parts. All intricately labelled, yet still so cumbersome to manage.

This Instructable takes a stab at solving the age-old problem of organizing a bunch of things. So without further ado, FindyBot3000.

Step 1: The Problem

Picture of The Problem

'nuff said.

Strickce1 month ago
Thanks for all of your help. I tried to follow the new update database instructions but I cannot get it to work. I installed Visual studio on my mac with no problems.
Then I went to the Azure portal the way I always do. I selected the database, and select to open it in Visual Studio. On the next screen, when I click "Open in Visual Studio", i get a Safari screen that says "Safari Cannot Open the Specified Address". The problem is "because macOS doesn't recognize internet addresses starting with vsweb://...

Any Idea how to get around this problem?

Should I install Visual Studio while in the Azure Portal?
Inventor22 (author)  Strickce1 month ago
New step is up showing how to manually connect to the SQL database from Visual Studio. Let me know how it goes.
Turns out that things don't work quite the same way on a Mac. None of the browsers work there. I couldn't get the second method to work either, so I punted. I now have a Windows10 running on my Mac, and everything works from the Windows machine. Your database modify instructions are exactly correct for Windows, and I have fixed my database and am happily on my way.

I really appreciate how responsive you have been. It's a pleasure working with someone who cares about their users.
Inventor22 (author)  Strickce1 month ago
Wish I could be of more help there, but I've never used a Mac, so at best any advice would be a shot in the dark. Good to know you were able to get things working with Windows 10 though, and have the database all fixed up.

When I was first starting with projects, I remember just how defeating it would be when I got hung up on a problem with no idea how to fix it, and how much I appreciated any help I got. Now, years later with a bit of experience under my belt, I get to pay it forward.
Inventor22 (author)  Strickce1 month ago
No problem. I think the issue you may be running into is Safari-specific. My browser is Chrome, running on a Windows machine and don't run into any problems. Some google-fu turned up this, it may help: https://discussions.apple.com/thread/5058055

If that doesn't, then the next easiest option is to try using a different browser, like Chrome or Firefox. Using the browser method here configures firewall rules and auto-fills some of the connection strings, making things a bit easier.

If you're not interested in installing another browser, hang tight as I'm working on another step to show how to manually connect to the database through Visual Studio.
Strickce1 month ago
I have it up and running, and have inserted items into several drawers. This is going to save me so much time. Even with labels on the drawers, it sometimes takes a long time to find an item. I do have a minor problem with the insert command. If I say: "ok google, insert 5 screws", google substitutes the letters five instead of the numeric 5. Other numbers work ok. When I send Quantity:five to Azure, it fails. Any idea why five is handled differently?
I can work around this by inserting 6 items and then "using" one, but that's a pain.
Inventor22 (author)  Strickce1 month ago
Right on! As for the issue with trying to insert five of an item, I've run into the same issue, sometimes with 'four' as well. I resolved to using the same solution you came up with as a quick fix.

I have no idea why the Google Assistant treats 'five' differently than other numbers -- not sure if even a Google engineer could tell you, neural nets are notoriously difficult to decode why they do something. My guess though, is because of the context in which 'five' can be used, for example: "Hey great work on that project, high five!". In this case, any user would want the words 'high five' in the sentence, and not 'high 5'. Use enough training data with "five" as the desired end result, as opposed to "5", and you'd get what we get here. Just a guess though.

There's a software fix I had in mind for this, but I haven't made time to get around to it. I'll post another comment if I implement a fix.
Thanks for the reply! I thought maybe I was either going crazy or that my KY accent was "confusing" Google. I have considered brute forcing any Quantity value to be numeric by just looking at google's published string and replacing the "for" or "five" with 4 or 5, respectively. I also found that I can type the request, using numbers, on google assistant app, and it works ok. Thanks again, and please don't think I am being negative in any way. I'm having so much fun working to make this as useful as I can.
Inventor22 (author)  Strickce1 month ago
You bet. A brute force solution was along the lines of what I was thinking too. There are unfortunately a lot of edge cases with this. On occasion, I've had google interpret a command like: "Insert seven one eighth inch pins" as "71 eighth inch pins" instead of the desired "7 1/8 inch pins". These kinds of cases are where things can get a lot more tricky.

Your idea with a mapping of "five" -> 5, "four" -> 4, etc. will work well enough to solve the bulk of these mistranslated scenarios though. Nice to hear you're having a good time with this project!
Strickce1 month ago
I have made a mistake and am in a position where I do not know what the database thinks is in several of the drawers. I moved some items getting ready for my full-blown inventory run, and goofed up. Now I need to know what the database has for those drawers, I logged on to the Azure portal and selected my database. I logged in as Admin and tried to open a new query, but the + sign does nothing. I did a query the other day, I thought the same way, but It won't work for me now. Im probably missing something obvious, so if you can help I would appreciate it.
Inventor22 (author)  Strickce1 month ago
Hey Strickce, I posted a new step outlining how to update the database. It's a lot more straightforward than working with SQL queries directly. Good luck, and let me know if you have any questions.
tctenng1 month ago
If 5v and ground is applied to each of the 14 strips on the left side, would just jumping the center pad on the strip suffice on the right side? I am not having a great deal of success in soldering the wires to the strips for some reason.
Inventor22 (author)  tctenng1 month ago
Yup, you're spot on. As long as each row strip is getting power, jumping only the middle data line every other row should be good to go. However, this does mean you'll need to connect up the power supply to test all the LEDs.
domnick00121 month ago
Wow! This is Crazy Idea
prnet2 months ago
Massive project! Absolutely brillant idea! Congratulation, I'm really impressed.
TaranA42 months ago
How does it know where items are?
Inventor22 (author)  TaranA42 months ago
Hi Taran, I use a database to store the location all the items which have been inserted using the "Insert Item" command. When the user asks for an item, "Ok Google, find yellow LEDs", the program extracts the words "yellow leds", then uses that as a 'key' to lookup the entry in the database. If an entry for the item is found, the coordinates of the box that the item is in are sent to the organizer, which then lights up the LEDs for that box.

It works similarly to how a dictionary works, but instead of finding the definition for a word or words, we get the coordinates of the box the item is in.
Strickce2 months ago
I have all the software connected, and am sending requests to Azure, but I have a syntax error in the command sent. I'm pretty sure that my IFTTT output is wrong, but I can't seem to get the correct stuff entered there. Azure tells me that there is a problem with the command input, and it seems to be associated with quotes. Could someone show me what a good "Insert Item" IFTTT event contains?
This is what I see on the serial monitor:

googleAssistantEventHandler event: Google_InsertItem, data: {“Info”:”red LEDs”,”Quantity”:50}
{“Info”:”red LEDs”,”Quantity”:50}
{"command":"InsertItem", "data":{“Info”:”red LEDs”,”Quantity”:50}}

The screen shots are the Azure error I get and my IFTTT screen


Update: I found the problem to be smart quotes in the data stream. I had to turn off Smart Punctuation in my iPad, and then re-enter the command to IFTTT. All is good now. Cannot wait to get the LED strips and actually build this. I am building a unit with 6-boxes instead of 4, and expect that to be a pretty easy change. I believe modifying the array dimensions should take care of most of the work.
Screen Shot 2019-05-09 at 9.22.17 AM.pngIMG_0015.JPG
Inventor22 (author)  Strickce2 months ago
Glad to hear you solved the problem Strickce. In my experience 90% of bugs with this project have been random unaccounted or inserted characters here and there which break the Json format one way or another. A good resource for analyzing and testing the Azure Function for any bugs / quality of output is to run the Azure Function locally with Visual Studio and send it requests using Postman (https://www.getpostman.com/). I've added the requests I used for testing to Github: (https://github.com/Inventor22/FindyBot3000/tree/master/Testing/Postman). Just import that file to Postman and you should be good to go.

As for 6 boxes (nice!), you're right you'll need to just update the array dimensions (In both the Particle Photon and the Azure Function code). There's also LED offset and LED width arrays (boxLedOffsetByColumnTop, boxLedWidthByColumnTop, etc.) which will need to be updated to account for the couple extra cabinets. The code is pretty janky around that so let me know if you need clarification on any of it.
Sebastian-Si2 months ago
I'v just tested it, works like a charm, but why MS Azure, it's a bit to expensive for project like this.
Have you wondered about local webservice?
Inventor22 (author)  Sebastian-Si2 months ago
Great to hear. I chose Azure for a couple reasons: I was already familiar with the Azure ecosystem, and I wanted to learn about Serverless Functions. I thought about running a local SQL (or other) database solution, or even rolling my own via EEPROM or an SD card, but I specifically wanted this device to be an IoT thing, so I quickly threw away those ideas.

In a future revision I may opt for on-chip storage and voice recognition, most likely running off a Raspberry Pi.
Hey Jude2 months ago
I love the tagging feature - as not only is it easy to forget what you called something at the first instance of assignment, but also for others to use the system it's essential!

I've watched a lot of Voice enabled tech, but this is actually one of the best I've seen.

Have you considered creating an App for this, with Google Home or Alexa, as it would simplify the code/SW part, and then the assembly could be easily done.

If for example the App allowed you to input the size/numbers/reference of the compartments, it would even allow easy updating. Granted this is a lot of effort on your part, but I'd part for the UI/UX of the App! I think it could be applied to many things, from food to pharma!

Well done on the win. First class!
Inventor22 (author)  Hey Jude2 months ago
Yeah, I had the same problem originally -- I'd forget what I called a part and end up wasting time manually looking for it. The tagging feature is key for letting others search for parts, as you mentioned.

I looked into creating a custom Google Assistant app, but the learning curve was a bit too steep for my timeline, and the integration with IFTTT was dead simple. So I made the trade off -- ease of use, for a little less functionality. A UI may come later down the road, including a little LCD to show part inventories and other debug info without the need to debug through a computer.

Thanks for the comment!
fernank22 months ago
Excelent
jatin72332 months ago
Can I use my android phone Google assistant instead of Google home??
And if yes, then how?
yes you can! When setting up the google assitant command on ifttt it will ask you to login to your google account, as long as google assistant on your phone is connected to the same account it will work no extra steps needed.
zpaeric2 months ago
This was an excellent project! I plan to make one as well, but I have many more parts than a set of single drawers can hold. If I built my shelves 5 deep, I can use flashing leds in the groups of 4 to identify which bin in that row contains the parts. No flash could be the front drawer and each flashing led would indicate drawer 2,3,4 and 5!

Thank you for the inspiration! Waiting for parts to arrive to get this started. I will post progress, and may have some programming questions to handle the 3D array...
Inventor22 (author)  zpaeric2 months ago
Thanks. That sounds great, a 3D setup would definitely be more compact. A bunch of different ideas could be used to represent the Z-axis, I like the flashing LEDs idea you mentioned. For sure, feel free to PM me or post any questions here.
science-tech3 months ago
great job bro
unbelivable! It is FANTASTIC! thank you so much for sharing, it is so cool! Have a great day :)
KitP63 months ago
The linked solid state relay is not one that can work on 5v DC switching input (They may have changed the product listing). :(
Inventor22 (author)  KitP63 months ago
Thanks, I've updated the Instructable with a new link and price.
KonradO63 months ago
Will it work with other languages supported by Google Assistant?
I’m from Brazil and I tried to test ifttt with google assistant in português and it wouldn’t work because ifttt was set to English. After changing my google assistant to English it worked , so I guess that if ifttt supports your language than you can use it
Thank you. So currently it's English only.
Or maybe there's an alternative to IFTTT.
Inventor22 (author)  KonradO63 months ago
Looks like IFTTT only supports English so far, as KonradO6 mentioned: https://help.ifttt.com/hc/en-us/articles/360001445233-Is-IFTTT-available-in-multiple-languages-
When using the ifttt app on my iphone and creating a google assistant applet there is a picker where eu can chooose one of 6 languages.
Can I use the google assistant app on an iPhone?
If I use a nodeMCU instead of photon do I need to change any code?
Inventor22 (author)  PortalGamesMais.3 months ago
Yeah, Google Assistant is available on iPhone: https://itunes.apple.com/us/app/google-assistant/id1220976145?mt=8

You will need to change some code around if you use nodeMCU, primarily in two main ways:
1. Use the Arduino-version of the neomatrix.h, Adafruit_GFX.h, neopixel.h, ArduinoJson.h libraries that I make use of in FindyBot3000.ino. These should auto resolve as you load the code into the Arduino IDE.
2. You can't use the Webhooks as implemented in the code, as they are configured through the Particle website (See Step 23: Software - Link Particle Photon to Azure Function). There will be a way to work around this though, but I couldn't tell you how off the top of my head, as I've never used nodeMCU. You might need to write custom HTTP requests. I did find this Instructable which looks promising: https://www.instructables.com/id/IoT-Air-Freshner-with-NodeMCU-Arduino-IFTTT-and-Ad/ and this: https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-arduino-huzzah-esp8266-get-started Those should help get you started.
Thanks.
Also can I use 5050 LED’s?
1-40 of 143Next »