Transcode Reolink streams for smaller recordings
While my video surveillance system covered all my home’s ingress areas, I did not have full coverage over my outside property. In order to remedy this situation, I installed some Reolink RLC-520A cameras since they allowed me to power them using power over ethernet, provided high resolution video streams and had decent online reviews. However, little did I know that their recordings would end up eating up most of my Frigate NVR’s allotted disk space. I needed to figure out a way to transcode these Reolink streams into smaller recordings.
Table of contents
About my Frigate setup
I have setup Frigate to perform continuous recording for every camera stream, even though it’s capable of saving recordings only when motion/objects are detected. I just enjoy being able to scrub through recordings to see the full timeline. Besides, I am more than happy to have just a few days worth of recordings.
Thus, I have allocated 500 GB of storage to store the recordings. This worked well prior to the installation of the Reolink cameras since I was able to retain all the 1080p recordings until they expired. Furthermore, the disk partition hosting the recordings is backed up on a daily basis using Bacula.
The problem with the new cameras
These cameras produce 2560×1920 video streams and their recordings were using around 2.5GB per hour, each! Some quick napkin math will tell you that such figures drastically reduced my data retention capabilities. Fortunately, Frigate will delete the oldest recordings to make room for new ones, but this ended up using way too much space for backups!
I had to remedy this situation.
Failed attempts to reduce storage usage
My first reflex was to ensure the cameras had audio recording disabled and I tried lowering the cameras’ stream resolution. Unfortunately, doing so seemingly reduced the field of view so this was out of the question.
I then attempted to setup Frigate’s go2rtc capabilities in order to transcode the video streams into a lower resolution. This appeared to do the trick, however the cameras suffered from connectivity issues after a few hours of streaming. Also, when looking at the birdseye view in Frigate, the streams were lagging by around 30 seconds. This resulted in significant gaps in my recording timelines and inadequate live-monitoring capabilities, so I reverted back to consuming the cameras’ RTSP streams.
Transcoding using MediaMTX
I came across MediaMTX while searching for other options for transcoding video streams. It checked a few boxes for me as it’s available as a container image, it is actively maintained and it is well documented. While MediaMTX is a full-featured media server/proxy, I really don’t need all of its bells and whistles. All I need it to do is transcode the Reolink camera streams and output them into another RTSP stream, so that Frigate can consume them and ultimately generate smaller recordings.
I went ahead and deployed it to my Kubernetes cluster as a single replica Deployment. I’m mounting the /mediamtx.yml configuration file as a ConfigMap object and I’m exposing the tcp/8554 RTSP service using a LoadBalancer, since Frigate is hosted outside of the cluster. The MTX_PROTOCOLS=tcp environment variable is needed because using UDP does not work when using the LoadBalancer. Make sure you use the bluenviron/mediamtx image having the -ffmpeg suffix as we will need it to capture input streams.
For MediaMTX’s configuration, I went ahead and disabled all services I didn’t need such as hls, rtmp, srt, etc. I next forced using the TCP protocol for RTSP. Then, I created paths for each camera I wanted to transcode and re-stream. You can see on line 267 that I’m defining the ext-front path, which causes MediaMTX to expose the following RTSP stream: rtsp://10.150.0.170:8554/ext-front – this is the URI Frigate will use to capture the transcoded camera stream.
Then, on line 268, I am defining the ext-front-original input stream. All the magic is happening on line 270, where ffmpeg captures one of my camera’s RTSP stream and performs the following notable tasks:
-anwill skip the inclusion of the audio stream.scale=1920:-1will resize the video stream to a width of 1920 and-1is used to adjust the height to keep the same aspect ratio as the input stream.-f rtsp rtsp://localhost:8554/ext-frontwill output the transcoded stream as an RTSP stream to the provided URI.
Here is the Kubernetes manifest containing all the objects:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mediamtx-deployment
namespace: mediamtx
spec:
replicas: 1
selector:
matchLabels:
app: mediamtx
template:
metadata:
labels:
app: mediamtx
spec:
volumes:
- name: mediamtx-configmap
configMap:
name: mediamtx-configmap
containers:
- name: mediamtx
env:
- name: TZ
value: "America/New_York"
- name: MTX_PROTOCOLS
value: "tcp"
resources:
requests:
cpu: "1"
memory: "1Gi"
image: bluenviron/mediamtx:1.8.3-ffmpeg
ports:
- containerPort: 8554
protocol: TCP
name: rtsp
volumeMounts:
- mountPath: /mediamtx.yml
name: mediamtx-configmap
subPath: mediamtx.yml
---
apiVersion: v1
kind: Service
metadata:
name: mediamtx
namespace: mediamtx
spec:
ports:
- name: mediamtx-rtsp
port: 8554
targetPort: rtsp
selector:
app: mediamtx
---
apiVersion: v1
kind: Service
metadata:
name: mediamtx-lb
namespace: mediamtx
annotations:
io.cilium/lb-ipam-ips: 10.150.0.170
spec:
ports:
- name: mediamtx-rtsp
port: 8554
targetPort: rtsp
selector:
app: mediamtx
type: LoadBalancer
---
apiVersion: v1
kind: ConfigMap
metadata:
name: mediamtx-configmap
namespace: mediamtx
labels:
app.kubernetes.io/name: mediamtx
data:
mediamtx.yml: |
logLevel: info
logDestinations: [stdout]
logFile: mediamtx.log
readTimeout: 10s
writeTimeout: 10s
writeQueueSize: 512
udpMaxPayloadSize: 1472
runOnConnect:
runOnConnectRestart: no
runOnDisconnect:
authMethod: internal
authInternalUsers:
- user: any
pass:
ips: []
permissions:
- action: publish
path:
- action: read
path:
- action: playback
path:
- user: any
pass:
ips: ['127.0.0.1', '::1']
permissions:
- action: api
- action: metrics
- action: pprof
authHTTPAddress:
authHTTPExclude:
- action: api
- action: metrics
- action: pprof
authJWTJWKS:
api: no
apiAddress: :9997
apiEncryption: no
apiServerKey: server.key
apiServerCert: server.crt
apiAllowOrigin: '*'
apiTrustedProxies: []
metrics: no
metricsAddress: :9998
metricsEncryption: no
metricsServerKey: server.key
metricsServerCert: server.crt
metricsAllowOrigin: '*'
metricsTrustedProxies: []
pprof: no
pprofAddress: :9999
pprofEncryption: no
pprofServerKey: server.key
pprofServerCert: server.crt
pprofAllowOrigin: '*'
pprofTrustedProxies: []
playback: no
playbackAddress: :9996
playbackEncryption: no
playbackServerKey: server.key
playbackServerCert: server.crt
playbackAllowOrigin: '*'
playbackTrustedProxies: []
rtsp: yes
protocols: [tcp]
encryption: "no"
rtspAddress: :8554
rtspsAddress: :8322
rtpAddress: :8000
rtcpAddress: :8001
multicastIPRange: 224.1.0.0/16
multicastRTPPort: 8002
multicastRTCPPort: 8003
serverKey: server.key
serverCert: server.crt
rtspAuthMethods: [basic]
rtmp: no
rtmpAddress: :1935
rtmpEncryption: "no"
rtmpsAddress: :1936
rtmpServerKey: server.key
rtmpServerCert: server.crt
hls: no
hlsAddress: :8888
hlsEncryption: no
hlsServerKey: server.key
hlsServerCert: server.crt
hlsAllowOrigin: '*'
hlsTrustedProxies: []
hlsAlwaysRemux: no
hlsVariant: lowLatency
hlsSegmentCount: 7
hlsSegmentDuration: 1s
hlsPartDuration: 200ms
hlsSegmentMaxSize: 50M
hlsDirectory: ''
hlsMuxerCloseAfter: 60s
webrtc: no
webrtcAddress: :8889
webrtcEncryption: no
webrtcServerKey: server.key
webrtcServerCert: server.crt
webrtcAllowOrigin: '*'
webrtcTrustedProxies: []
webrtcLocalUDPAddress: :8189
webrtcLocalTCPAddress: ''
webrtcIPsFromInterfaces: yes
webrtcIPsFromInterfacesList: []
webrtcAdditionalHosts: []
webrtcICEServers2: []
webrtcHandshakeTimeout: 10s
webrtcTrackGatherTimeout: 2s
srt: no
srtAddress: :8890
pathDefaults:
source: publisher
sourceFingerprint:
sourceOnDemand: no
sourceOnDemandStartTimeout: 10s
sourceOnDemandCloseAfter: 10s
maxReaders: 0
srtReadPassphrase:
fallback:
record: no
recordPath: ./recordings/%path/%Y-%m-%d_%H-%M-%S-%f
recordFormat: fmp4
recordPartDuration: 1s
recordSegmentDuration: 1h
recordDeleteAfter: 24h
overridePublisher: yes
srtPublishPassphrase:
rtspTransport: automatic
rtspAnyPort: no
rtspRangeType:
rtspRangeStart:
sourceRedirect:
rpiCameraCamID: 0
rpiCameraWidth: 1920
rpiCameraHeight: 1080
rpiCameraHFlip: false
rpiCameraVFlip: false
rpiCameraBrightness: 0
rpiCameraContrast: 1
rpiCameraSaturation: 1
rpiCameraSharpness: 1
rpiCameraExposure: normal
rpiCameraAWB: auto
rpiCameraAWBGains: [0, 0]
rpiCameraDenoise: "off"
rpiCameraShutter: 0
rpiCameraMetering: centre
rpiCameraGain: 0
rpiCameraEV: 0
rpiCameraROI:
rpiCameraHDR: false
rpiCameraTuningFile:
rpiCameraMode:
rpiCameraFPS: 30
rpiCameraIDRPeriod: 60
rpiCameraBitrate: 1000000
rpiCameraProfile: main
rpiCameraLevel: '4.1'
rpiCameraAfMode: continuous
rpiCameraAfRange: normal
rpiCameraAfSpeed: normal
rpiCameraLensPosition: 0.0
rpiCameraAfWindow:
rpiCameraTextOverlayEnable: false
rpiCameraTextOverlay: '%Y-%m-%d %H:%M:%S - MediaMTX'
runOnInit:
runOnInitRestart: no
runOnDemand:
runOnDemandRestart: no
runOnDemandStartTimeout: 10s
runOnDemandCloseAfter: 10s
runOnUnDemand:
runOnReady:
runOnReadyRestart: no
runOnNotReady:
runOnRead:
runOnReadRestart: no
runOnUnread:
runOnRecordSegmentCreate:
runOnRecordSegmentComplete:
paths:
all_others:
paths:
ext-front:
ext-front-original:
runOnInit: >
ffmpeg -rtsp_transport tcp -i rtsp://redacteduser:[email protected]/h264Preview_01_main -pix_fmt yuv420p -c:v libx264 -preset ultrafast -b:v 600k -max_muxing_queue_size 9999 -an -filter:v scale=1920:-1 -f rtsp rtsp://localhost:8554/ext-front
runOnInitRestart: yesStability, performance and storage usage
By incorporating MediaMTX between Frigate and my cameras, video recordings are now using around 300 MB of disk space per hour for the Reolink cameras when using the transcoded video streams. This way, I can easily fit several days worth of recordings on Frigate’s allotted storage space!
As for stability, there are seemingly no gaps in the camera timelines and both Frigate and MediaMTX logs show no signs of connectivity issues.
With regards to performance, MediaMTX is using around half a CPU core and 500 MB of RAM to transcode 2 camera streams.
While MediaMTX is not using GPU acceleration to perform the decoding and encoding, the Kubernetes nodes are hosted in Proxmox using the x86-64-v3 emulated processor type. I suspect that this processor type may allow for some form hardware offloading when decoding/encoding video streams, but still, this is very decent resource usage.
When watching the Birdseye view in Frigate, I can still see a ~10 second lag when comparing the camera’s timestamp with the actual time, but this is acceptable.
Conclusion
Introducing MediaMTX as a middleman between Frigate and my Reolink cameras to transcode their video streams proved to be an excellent solution to produce smaller recordings. I can now retain around two weeks of continuous video recordings on Frigate’s 500 GB drive.
I suspect there may be a way to replicate what MediaMTX does within Frigate’s go2rtc configuration but considering how little resources the former uses and how easy it is to configure, I’m happy with the outcome.
