Save both audio and video to one file using GStreamer from camera and microphone

Here it is:

gst-launch-1.0 -e autovideosrc ! queue ! videoconvert ! mkv. autoaudiosrc ! queue ! audioconvert ! mkv. matroskamux name=mkv ! filesink location=test.mkv sync=false

This is how it works:

We get video feed from webcam using autovideosrc.
Next, we place that feed in it's own thread by outputing it into queue.
Video feed from queue goes to videoconvert where it is converted (I'm not sure is this step necessary).
Finally, video feed goes into element named mkv, which is of type matroskamux. This element converts video feed to Matroska.
After that we get audio feed using autoaudiosrc.
We place feed from autoaudiosrc in another thread using queue.
From queue audio goes to audioconvert where it is converted (also not sure if necessary).
Converted audio goes to the same matroskamux element mkv. Here audio feed is converted to Matroska.
Since both audio and video feed go to the same element, they are merged into one Matroska feed.
At last, that Matroska feed is saved to a file using filesink. You'll probably have to set sync=false if your computer isn't a beast.

That's it. Also, if someone's wondering why I wrote in the question that this command doesn't record audio, it's because I forgot to turn mic on.

Hanlon

Posted 2018-10-18T16:48:37.923

Reputation: 431

Hi there. I'm using your command inside python gstreammer, but it is not working when I plug some audio / video src to the pipeline insteqad autovideosrc. Have you ever used python for this? – DGoiko – 2018-11-26T10:12:34.163

@DGoiko Hello! I didn't try it using Python but I did try it in C and it worked. – Hanlon – 2018-11-28T15:47:23.007

I've got it connected to an appsrc (tomething similar to this: https://github.com/streamlink/streamlink/blob/master/examples/gst-player.py , but using uridecodebin instead of playbin and some custom bins, included one based in your pipeline to record) and it stops at the begining. The player itself is fine: If I connect audio and video tees with queues to autoaudiosrc + autovideosrc + flac audio recording everything works fine, but if I put the video queue it freezes at the beginning. Debug 3 shows nothing... Its strange.

– DGoiko – 2018-11-29T03:39:34.430

Could you post some code? Preferably you could open a new question, but copying code to Pastebin is fine too. Just be sure to give me link here. – Hanlon – 2018-11-30T09:05:36.080

https://stackoverflow.com/questions/37416015/gstreamer-write-both-video-and-audo-streams-into-a-single-mp4-container-after-c – music2myear – 2018-10-18T16:50:31.583

@music2myer So I have to record video and audio separately and then concatenate them? Can I not just save them to a same file at once (while recording)? – Hanlon – 2018-10-19T07:44:08.727

I don't know the features of gstreamer, but the answer for the question I linked suggested to read documentation on gst-launch, which while offering concat features, I suppose may also offer real-time audio & video capture, as suggested by the answer you figured out below. Glad you were able to get it all working. – music2myear – 2018-10-19T15:28:07.773

Save both audio and video to one file using GStreamer from camera and microphone

Answers