👨‍💻 I'm in search of a remote Ruby/Rails Developer position. Hire me.

Linking a Webcam Directly to Rails' ActiveStoragePhoto by Laura Boccola

ActiveStorage is awesome. For all the times I’ve seen it being used, it was for direct file uploads where the user clicks a file field, a window pops up and a file is selected–nothing new.

Not long ago, I was tasked with developing a feature to allow users to upload pictures, not from the traditional sense that we already know but from the webcam. This was novel.

Note: I have built a working version of this here and also posted the code on GitHub for your perusal.

Getting the Frontend Sorted Out

First, we need access to the user’s webcam. This can be achieved with JavaScript through the navigator.mediaDevices API. The navigator.mediaDevices read-only property returns a MediaDevices object, which provides access to connected media input devices like cameras and microphones.

if (video) {
    navigator.mediaDevices.getUserMedia(constraints).then((stream) => {
        video.srcObject = stream
    })
}

This streams a live feed from the webcam. But we want pictures, to do this we set a canvas up and draw the content of the stream on it, this could be at any time, to capture just a frame. HTMLCanvasElement provides a toDataURL() method which returns a data URL containing a base64 encoded representation of the image which we’ll get to in a bit.

The data URL that we get from the stream is a very long string. Initially, when I was thinking of how to implement this I thought it’d be OK to submit strings through params but quickly retracted this idea! Reason being that different web servers may have a maximum allowed size of the client request body set by default, in such cases if we sent a data URL that happens to be too big, we might get a response status code of 413 Payload Too Large which browsers may not be able to correctly display.

A hidden field, on the other hand, allows sending of data that cannot be seen or modified by users when a form is submitted, plus an added advantage of having no technical limit (the perfect option for a very long data URL) in browsers. For these reasons, it’s safe to send the data URL through a hidden field on the form to the controller like this:

<%= form.hidden_field :player_picture, value: @player.player_picture %>.

If for some reason, you’re paranoid about security and what can come through the hidden field, on the backend you can always filter what gets pushed through, after all, it’s the internet where all sorts of weirdos cohabitate with us.

player_picture would be a virtual attribute on the model since we’re not interested in making this persistent but just need something to hold the data URL for the image taken from our webcam so we can pass it on to ActiveStorage. The player_picture has to be accepted in the params inside the controller.

For the frontend, I have set up everything in webcam.js. One other important thing worth noting is after getting access to the webcam, we then need to draw a frame from the stream to the canvas, which happens with the following code:

  if (snapButton) {
    snapButton.onclick = function () {

    // snip!

      canvas.getContext('2d').drawImage(video, 0, 0)

      var dataUrl = canvas.toDataURL('image/jpeg')

      document.getElementById("shot").src = dataUrl

      hiddenPlayerPicture.value = dataUrl

    // snip!

    }
  }

The relevant part in the code above is hiddenPlayerPicture.value = dataUrl where we set the value of the hidden field to dataUrl which is the data URL of the frame we grabbed from the webcam stream.

Attaching Webcam Images to ActiveStorage

I haven’t made mention of setting up Rails ActiveStorage because it’s assumed this has been done already.

In our model we can set the virtual attribute I mentioned earlier like so:

# /app/models/player.rb

attribute :player_picture, :string, default: ''

and then have the controller accept it:

# /app/controllers/players_controller.rb

  def player_params
    params.require(:player).permit(
     # other params
     :player_picture)
  end

Now that we have the captured frame coming in through the form to the controller, we can attach that data to ActiveStorage. For our use case here we’re only interested in attaching an image to a Player through the update action. Now would be a good time to abstract this functionality to a service which looks like this:

# /app/services/picture_attachment_service.rb

class PictureAttachmentService
  class << self
    def attach(model, picture)
      base_64_image = picture.gsub!(/^data:.*,/, '')
      decoded_image = Base64.decode64(base_64_image)

      model.picture.attach(
        io: StringIO.new(decoded_image),
        filename: "player_picture_#{unique_string}.jpeg"
      )
    end

    private def unique_string
      SecureRandom.urlsafe_base64(10)
    end
  end
end

This is the crux of the picture attachment to ActiveStorage. A handful of things are happening here. Remember the base64 encoded representation of the image we mentioned above that comes from toDataURL()? This takes the form:

data:[<mediatype>][;base64],<data>

There are fours parts to this; a prefix (data:), a MIME type indicating the type of data, an optional base64 token if the data is non-textual, and the data itself.

An example of such representation is:

data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAMCAgICA...42vYWS34f/9k=

We just want the <data> part of this and that’s what base_64_image = picture.gsub!(/^data:.*,/, '') does for us. Then since it’s encoded, we need to decode it with decoded_image = Base64.decode64(base_64_image). This gives us a decoded string that we can work with.

One thing I love about ActiveStorage is the fact that it allows us to attach IO objects. An example provided by Rails Guides looks like this:

@message.image.attach(io: File.open('/path/to/file'), filename: 'file.pdf')

Luckily for us, we can have a StringIO which in our case is the decoded base64 data. That’s it. We’ve successfully built the parts that we can connect to attach a frame from a stream (picture) from a webcam.

Wait. Not so fast.

We forgot to call this service in the update action of our controller.

This is how our controller should look like now:

# /app/controllers/players_controller.rb

def update
  PictureAttachmentService.attach(@player, params['player']['player_picture'])

  respond_to do |format|
    # some stuff
  end
end

OK, now we’re done.

Attaching Webcam Videos to ActiveStorage

The process for attaching a webcam video is similar to the one done for images. Except, in this case, there’ll be some more work. But the most important thing to remember here is after setting up the UI with play, pause controls, we can get a video of MIME type video/webm through the very same navigator.mediaDevices we used earlier. But bear in mind that if the MIME type for the video is not set correctly on the server, the video may not show or show a grey box containing an X (if JavaScript is enabled). I don’t think though, that there would be an issue with video/webm.

MDN Web Docs has a very good tutorial on how to get the UI right with video recording.

Once the UI bits are in place, all it takes is to display the video in your view with something like this:

<video width="500" height="300" autoplay loop="true">
  <source src="<%= url_for(@player.video) %>"
     type="video/webm">
</video>

Where @player.video is the video/webm you’d get from navigator.mediaDevices.

We can edit our service to look like this accordingly:

class VideoAttachmentService
  class << self
    def attach(model, video_path)
      model.picture.attach(
        io: File.open(video_path),
        filename: "player_video_#{unique_string}.webm"
      )
    end

    private def unique_string
      SecureRandom.urlsafe_base64(10)
    end
  end
end

There are a few things we could have talked about; error handling in case something goes wrong with our attachment service, setting up a background job for the service with Sidekiq or Resque, security, refactoring, web server limits, tests among others but those are beyond the scope of this post. The main plan has been executed. Everything works, everyone is happy.

I believe ActiveStorage is capable of a lot more, one example is analyzing videos. This is a lot to appreciate.