Frankie the Slipmat Native Video Encoder is a purpose-built RTMP encoder server. This document is an in-depth guide of the internal workings of Frankie. If you are looking for basic setting up guide, read Setting Up Fankie.
Understanding the encoding process
It’s important to have a high-level understanding on how the streaming engine works in order to be able to test it properly. (Frankie in the picture is named
Slipmat RTMP live event is built with of three distinct servers; 1) Streaming server (Frankie), 2) Application server, and 3) Frontend Web Server. There is also possibility to use any third-party CDN-service, but this doesn’t affect the video flow.
The live event page consists of two distinct pieces; 1) event application code and 2) event video. Application code is always served from the main application server, while the event video might be served from YouTube, any third-party RTMP-server, or in this case, from Slipmat Frontend Web Server.
(Note that Frankie is physically located in Germany. If you have audience for example in the west coast of US or in Australia, they might encounter buffering just because of physics of the Internet; it sometimes takes longer than 2 seconds to download a short video segment Germany to the other side of the planet, because the packets on the internet travel various routes and sometimes get lost. If you or your listeners encounter buffering, please report it so we can investigate the situation and plan for proper CDN techniques. Optimally we would always use CDNs and route the video to the listeners from nearest available server but it costs thousands of euros per month and we don’t currently have the money for it.)
Encoding process flow
When you start your encoder software (OBS, MimoLIVE, etc), it first connects to Frankie for a handshake. The streaming server passes your secret streaming key to Slipmat Application server and validates the key. If the key is invalid, or if your DJ profile is for example deactivated or deleted, the stream engine will not start and the server drops the connection.
Note: Some software (like mimoLive) try to auto-connect immediately which can lead to endless connect-handshake-drop-reconnect -loop. If this happens, check that your stream key and ingest server URL is valid. If this still happens, report the bug here.
After a successful handshake, the server returns a valid public key for the streaming engine that is used to generate your public stream URL. This process starts the actual encoding process and the public video URL is created.
HLS video is adaptive bitrate system that consists of a manifest (m3u8-file) of one or more available bitrate variant playlists that themselves consist of short 2 second video segments. Slipmat video player expects at least 6 segments before the video starts playing so your theoretical minimum video lag is about 12 seconds. In practise this usually ends up being something around 13-15 seconds, depending on your video content, the color of electricity and the alignment of the planets in the solar system.
Do note that because the playlist is at minimum 12-seconds long, errors can take up to 12 seconds to show up in the video player. Also, if your streaming software or internet connection hangs more than 5 seconds, the encoding process is killed and will need to be started again.
In practice, if you have buffering issues or other problems with your software or internet connection, it’s best to stop streaming, wait for ~10 seconds and then start streaming again. This fully empties the server buffer and restarts the encoding process cleanly so any previous problems just fade away.
Your listeners in the meanwhile only see buffering for few seconds, and if everything works as expected the video just starts playing after the encoder gets more video segments ready. If the player doesn’t receive enough segments in time, it waits for a couple of seconds and then refreshes itself. It does this several times before giving up so you have plenty of time to fix your issues. If you have a major issue during a live event and the stream has been dropped for a long time, you can fore-refresh the live page for all listeners. So, as soon as you get the video going again, manually refresh your own live page (on Mac Command-R, on Windows F5 or something like that), and when the video starts working, click the force refresh button and all your listeners will get the working video as well.
Different encoding application endpoints
There are two different encoder applications that you can use and test. These are designed for different situations so it’s good to know available options and understand which suits best for your needs.
You select the app in use by switching out the end part of your ingest server URL:
Stream specs: 256k or 320k AAC audio, 2500kbps/720p video, transcodes 3 video streams (300k, 850k, 2500k) with copied audio (no transcoding).
This is the default which you should be using most of the time. The 720p video at 2500kpbs is a very good compromise between bandwidth and image quality. You are most likely using cheap cameras with low light so the image quality is pretty poor to begin with, so there is no sense in trying to go overboard here. Audio quality is superb but only if you set your source to 256 or 320 AAC @ 44.1. If the quality is set correctly in your streaming app, the audio won’t be touched at all and your listeners will get the best sound possible – even with the lowest 300k/240p video setting. Note that if you send incorrect audio bitrate, the playlist bitrate calculation is not correct and your listeners may encounter issues.
Video playlist length is set to 12 seconds, which is 6 x 2s segments (Apple dictates minimum of 6 segments for playback with iOS devices). Your theoretical video lag is about 13 seconds.
Stream specs: 128k AAC audio, 750kbps/480p video, transcodes 2 video streams (300k, 750k) with copied audio (no transcoding).
This is an alternative encoder that is suited best for mobile and other low-bandwidth situations. If you or most of your listeners are either in spotty 3G / 4G environment or physically very far from Finland, you might want to use this option.
This encoder writes longer 4s segments which gives the clients 4s to download each upcoming segment and also the server time to process it. Video playlist length is set to 24 seconds, which is 6 x 4s segments. Your theoretical video lag is about 25 seconds.
- Flaky connections may break the encoder. The encoder relies on steady stream that is exactly in the promised limits. In real life internet rarely works perfectly so your streaming software might send 2500 kbps stream 400 kbps in one second and 3400 kbps in next. This may result in missed video segments and errors in encoding. Missed segments in the playlist result in buffering for the listener and too many errors may result in non-working video. If you encounter this, stop the stream, lower your KBPS rate and start stream again to kill the encoder.
- Non-standard streaming rates will break the encoder. Please stay in the given limits with both video and audio. If you try to push 5000kbps stream into the streamer, it only stresses the encoder more and it makes the video quality worse, not better. The encoder is limited to given kbps-rate and it will enforce it, there is no way to raise it from streamers end. All encoding processes are logged and if we see unnecessary non-standard rates, you will be locked out of using Frankie. (Lower than optimal rates are OK if the streamer has issues but they’re not optimal either as the video frames will not be consistent.)
- Make sure to set your output exactly 2500kbps video + 256k audio or 750k video + 128k audio, depending which application you use. If the incoming stream is optimal, the encoder needs to do less work and the end result will be optimal.
- Also make sure that your encoder sends a keyframe exactly every 2 seconds. Anything else is sub-optimal and may break the video. More than keyframe every 2 seconds will waste your bandwith, less will make your video look bad or not work at all.
- Set default video URL and default event image in your DJ profile to make event creation smoother. You can always change things when you need to, but if you have defaults set, your normal workflow gets way faster and easier.
- Spin test events with no audience just to get to know the workflow. Mark your test events as private so they won’t show up anywhere and spoil your statistics. Test events are a great way to make sure everything works before you start an actual show for your fans. You can test different settings and share the event link to friends (or here with this group!) to get feedback and help.
- Note also that your streaming computer affects the quality and stability of the stream. We’ve found with practical testing that 2500 kbps bitrate seems to be optimal for 720p video for the CPU. If you need to stream lower bitrates, note that this might tax your computer more (this is counter-intuitive but true: the computer needs to do more work to pack the 720p image into smaller file size) Ideally you should stream from a separate machine than you run the DJ software.
Important last note
Do report all and any issues you encounter! Your testing is pointless if you encounter errors and we don’t get a chance to learn from them and fix them. No excuses here; be diligent and remember to post all and any errors you encounter. Remember to mention the event time and name/URL as well so we know where to look.
Feature requests and ideas are also welcome. Again, this is your chance to make a difference; if you have something to say, please say it!
Remember also that disagreeing and debating is also totally fine. So far all of the design decisions around the streaming engine have been very one-sided, so if you have preferences or opinions, don’t be afraid to tell them.