JSFoo 2013

All about being creative with JavaScript

Bot using NodeJS and the HTML5 Audio API

Submitted by Ameya Karve (@ameyakarve) on Aug 4, 2013

Section: Crisp Talk Technical level: Intermediate Status: Confirmed & Scheduled


To build a robot that is controlled using a NodeJS server. The bot uses the HTML5 Audio API to actuate it's motors. It can be run using any mobile device that can connect to the internet, and has a web browser and an audio port.


This is a modification of one of my earlier hacks. The idea is to control a robot using the sound output from a mobile device. The sound output is controlled through a web page on the mobile device browser, that talks to a NodeJS server. The communication can be realtime when supported by the device using a WebSocket connection; I have also written a long-polling fallback mechanism. I have tested the hack using an LG P500, a Nexus 4 as well as an iPhone 4; It should seamlessly work with other devices otherwise.
The hack does not require a circuit board like an Arduino or a Raspberry Pi device; I can share the circuit details, as well as places where the components could be procured in Bangalore or Chennai.


Knowledge of NodeJS and Socket.IO

Speaker bio

My name is Ameya Karve. I am a final year student at IIT Madras. I have been playing around with JavaScript for a while now. You can see some of my work at my GitHub profile; my handle is ameyakarve.



  • Shekhar Upadhaya (@zerobyte) 6 years ago

    looking forward for this session!!

  • Aravind Krishnaswamy (@arg0s) 6 years ago

    IMHO, your outline seems a bit misleading:
    a) You say youve written a long polling fallback mechanism. As far as I can tell from your code, you are piggybacking on socketio’s inbuilt transports. I dont see anything added - can you clarify?
    b) When you said you were using HTML5 Audio API, I expected webaudio and recording - but it looks like you are only using the audio tag for playback. Can you clarify?
    Not trying to take anything away from what you’ve done - just trying to understand.

    • Ameya Karve (@ameyakarve) Proposer 6 years ago


      The code went through some changes since I first posted it. I realise now that it is potentially misleading; it works all the same though. I will try to like clarify the points
      1. The long polling: I had started with a long-poll, and ended up implementing the web-socket much later; For the long polling hack, I essentially rendered separate pages using my node server depending on a context. I could simply have modified my socket code though, now that I think of it.
      2. The code was initially done with the audio API, but there were some playback errors I was getting on Apple devices; hence I shifted to simple Audio tags and handled them with JS

      The essence of the hack was that the Audio output is a legitimate output signal that most devices can use, and when manipulated rightly, it can be used to do stuff. I hope I am clear; do let me know if you have other queries


  • Om Shankar (@omshiv) 6 years ago

    I am trying to build a DEMO app for JSfoo talk. Requires socket. But I will be involving audiences and so cannot host on localhost.

    Need a hosting server that supports Sockets. Heroku doesn’t ! Do u know of any Socket supported cloud hosters?

    Also, saw that you were using longpolling. If your demo is hosted locally, then you can very well use real WebSockets than long polling.

    • Ameya Karve (@ameyakarve) Proposer 6 years ago (edited 6 years ago)

      Sockets will work over Wifi. If both your server and the client are connected to a single Wifi network, then simply point the web browser of your client to the IP address of your server.

      For example, if my laptop (used to host the node server) is at

      From my own computer, I can access the server at localhost

      From a mobile phone connected to the same Wifi network, just enter in the address bar. It should work fine.

      Remember, that while launching socket.io, write var socket. = io.connect() instead of var socket = io.connect('localhost')

      Also, get the ports right; I think socket.io uses 80 by default, so it won’t matter.

      Tell me if it works out; it did on my Router

      Mail me at ameya [dot] karve [at] gmail [dot] com if you need more help

      • Om Shankar (@omshiv) 6 years ago (edited 6 years ago)


        2 things though:

        • io.connect is when you are relying on the client side library served in the browser while you are using socket.io in Node. I am working with real WebSockets API in HTML5 (The native one): var WS = new WebSocket('ws://your-socket-server...');

        • I have tried using localhost in BangaloreJS for my demo, did not work due to NAT firewall.

        • Ameya Karve (@ameyakarve) Proposer 6 years ago

          I assumed you will be using socket.io . There must be a workaround with using websockets directly too, because socket.io does invoke them. Maybe you can look at the socket.io implementation; you might find something there

          I am not sure about the firewall issues; I hope it won’t interfere much though


          • Om Shankar (@omshiv) 6 years ago

            Ofcourse I am using socket.io. But your implementation is old and different.

            The way it works is, you use socket.io in your Node App. Assuming that your browser does not support WebSockets, you use a script tag in your html with src pointing to /socket.io/socket.io.js.

            But this is an implementation for non-WebSocket supported browsers, in which case, socket.io at your node.js server will fall-back to longpolling; which is what you are using in ur code

            If you are using your own machine, you can very well use HTML5 WebSockets in browser, without using client-side script tag for the same; u can show the demo on Chrome. Which has a native support.

            At the server-side, you would have to change from require('socket.io'), to require('websocket.io').

            I just wanted to know if you knew any cloud servers that have WebSocket support for Node.

            • Ameya Karve (@ameyakarve) Proposer 6 years ago (edited 6 years ago)

              About the hosting, I believe AppFog had sockets supported. They took it down for a while though; Seems like it is back, but I am not sure of they still have sockets.

              I didn’t quite get what you were trying to say. If I simply use var socket. = io.connect() on my client, he will talk to the Node server using WebSockets if supported, and fallback to longpolling if it fails.

              On the server, all I do is

              var app = express(), 
              server = require('http').createServer(app),
               io = require('socket.io').listen(server);

              assuming that you are using express on top of Node. Socket will take care of whether the connecting client has WebSocket support or not right?

              So, If I run this server locally, and my laptop running the server is connected to a WiFi network, I can simply point my mobile browser ( client supporting WebSockets) to the relevant IP address. He will open a socket connection with the server. I have verified this. I am sure you could use it too.

              Again, I possibly interpreted your question wrong; Did you want to allow only those devices that support WebSockets by chance?

              • Om Shankar (@omshiv) 6 years ago (edited 6 years ago)

                Socket.io is a node.js + browser library to work with WebSockets cross-browser.

                Using socket = io.connect() at client is the method of the library, when you are using socket.io at server. This is for implementing WebSockets in browsers that do not have support for WebSockets, where it would fallback to long polling.

                Plus, it works like native in browsers that support WebSockets.

                When you are giving a demo locally on ur machine (hence having full server control), u can choose the browser, and hence no need to go with socket.io.

                One can simply use websocket.io at server (smaller foot-print) and no library in front-end (since the method is available natively) !

  • Ashwin Acharya (@ashwnacharya) 6 years ago

    Very good session.

Login to leave a comment