threefive

README

threefive

threefive is a SCTE-35 Decoder / Parser library in Python3 .

threefive references the 2020 SCTE-35 Specification.

threefive is Easy to use. Check out these Examples

threefive decodes SCTE-35 from MPEG-TS video files and streams.

threefive decodes SCTE-35 from Base64, Hex, and Binary encoded strings.

threefive decodes SCTE-35 from streams transcoded by ffmpeg as Data: bin_data ([6][0][0][0] / 0x0006).

threefive is now testing a Golang version too..

---

• Versions and Releases

• Install threefive

• Fast Start

• Easy threefive (Using the decode function )

  • Mpeg-TS Streams
  • Mpeg-TS Streams over HTTPS
  • Base64, or Hex encoded strings
  • Hex Values and Integers

  ---

• Advanced threefive

  • Cue Class
  • Stream Class

  ---

• Examples

  • Parsing SCTE35 from MPEGTS over HTTPS (New and Improved!)

  • Parsing HLS Manifests with threefive (New Code!)

  • Stream.decode_proxy() Example

  • Show preroll

  • Upids_with Custom Output

  • Multiple Segmentation Descriptors

  • Multicast Stream Server and Client for SCTE-35

  • All Examples

  ---

• threefive Spotted in The Wild

• ffmpeg and SCTE35 and Stream Type and threefive

• Code of Conduct.

• Issues and Bugs and Feature Requests

---

Versions and Releases

---

Odd numbered Versions are Releases.

Even numbered Versions are Testing Builds and may be Unstable.

a@fumatica:~/threefive$ pypy3

Python 3.6.12 (7.3.3+dfsg-3, Feb 25 2021, 22:28:03)
[PyPy 7.3.3 with GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> from threefive import version
>>>> version()
'2.2.82'

Install

---

• Requires python 3.6+ or pypy3

threefive runs 3x Faster on pypy3

• install from pip (recommended)

$ pip3 install threefive

# for pypy3
$ pypy3 -mpip install threefive

#If you don't have pip installed, try this.
$ pypy3 -mensurepip install pip 

---

• install from git

$ git clone https://github.com/futzu/threefive.git

$ cd threefive
$ make install

# for pypy3 
$ make pypy3

---

Easy threefive

The decode Function

• src decode.py
• threefive.decode is an all purpose function to decode SCTE 35 messages from a file or string.

import threefive

• MpegTS

threefive.decode('/path/to/mpegwithscte35.ts') 

• MpegTS over http and https

threefive.decode('https://futzu.com/xaa.ts') 

• Base64

mesg='/DA4AAAAAAAA///wBQb+AAAAAAAiAiBDVUVJAAAAA3//AAApPWwDDEFCQ0QwMTIzNDU2SBAAAGgCL9A='
threefive.decode(mesg)

• Hex String

hexed = "0xFC301100000000000000FFFFFF0000004F253396"
threefive.decode(hexed)

• Hex Values( New! )

raw_hex = 0XFC301100000000000000FFFFFF0000004F253396
threefive.decode(raw_hex)

• Integers ( New! )

big_int = 1439737590925997869941740173214217318917816529814
threefive.decode(big_int)

• Read a string directly from a file encoded in Base64, Binary or Hex

$ cat cue.dat
   /DBIAAAAAAAA///wBQb+ek2ItgAyAhdDVUVJSAAAGH+fCAgAAAAALMvDRBEAAAIXQ1VFSUgAABl/nwgIAAAAACyk26AQAACZcuND

from threefive import decode

decode('cue.dat')

---

Advanced threefive

---

Cue Class

class Cue(builtins.object)
 |  The threefive.Splice class handles parsing
 |  SCTE 35 message strings.
 |  Example usage:
 |  
 |  from threefive import Cue
 |  
 |  Base64 = "/DAvAAAAAAAA///wBQb+dGKQoAAZAhdDVUVJSAAAjn+fCAgAAAAALKChijUCAKnMZ1g="
 |  scte35 = Cue(Base64)
 |  scte35.decode()
 |  scte35.show()
 |  
 |  Methods defined here:
 |  
 |  __init__(self, data=None, packet_data=None)
 |      data may be packet bites or encoded string
 |      packet_data is a dict passed from a Stream instance
 |  
 |  __repr__(self)
 |  
 |  decode(self)
 |      Cue.decode() parses for SCTE35 data
 |  
 |  get(self)
 |      Cue.get returns a dict of dicts
 |      for all three parts of a SCTE 35 message.
 |  
 |  get_descriptors(self)
 |      Cue.get_descriptors returns a list of
 |      SCTE 35 splice descriptors as dicts.
 |  
 |  get_json(self)
 |      Cue.get_json returns the Cue instance
 |      data in json.
 |  
 |  mk_info_section(self, bites)
 |      Cue.mk_info_section parses the
 |      Splice Info Section
     of a SCTE35 cue.
 |  
 |  show(self)
 |      Cue.show pretty prints the SCTE 35 message
 |  
 |  to_stderr(self)
 |      Cue.to_stderr is a Wrapper
 |      for printing to sys.stderr
 |  
 |  ----------------------------------------------------------------------

---

• src cue.py
• The threefive.Cue class decodes a SCTE35 binary, base64, or hex encoded string.
• threefive.Cue provides several methods to access the parsed data.

from threefive import Cue

b64 = "/DBIAAAAAAAA///wBQb+ek2ItgAyAhdDVUVJSAAAGH+fCAgAAAAALMvDRBEAAAIXQ1VFSUgAABl/nwgIAAAAACyk26AQAACZcuND"

cue = Cue(b64)
cue.decode()

• A decoded Cue instance contains:

  • cue.info_section

    • 1 threefive.SpliceInfoSection()

  • cue.command

    • 1 of these commands:

      • threefive.BandwidthReservation()
      • threefive.PrivateCommand()
      • threefive.SpliceInsert()
      • threefive.SpliceNull()
      • threefive.TimeSignal()

  • cue.descriptors

    • a list of 0 or more of these descriptors :

      • threefive.AudioDescriptor()
      • threefive.AvailDescriptor()
      • threefive.DtmfDescriptor()
      • threefive.SegmentationDescriptor()
      • threefive.TimeDescriptor()

  • crc

  • 'When parsing SCTE35 Cues from MPEGTS streams, threefive attempts to include as many of the following as possible.'

    • pid of the packet
    • program of the pid
    • pts of the packet
    • pcr of the packet

---

• All instance vars can be accessed via dot notation.

>>>> from threefive import Cue
>>>> cue = Cue(b64)
>>>> cue.decode()
True
>>>> cue.command
{'command_length': 5, 'command_type': 6, 'name': 'Time Signal', 'time_specified_flag': True, 'pts_time': 22798.906911}
>>>> cue.command.pts_time
22798.906911
>>>> 

• call one or more of these methods after decode.

Cue Method	Description
cue.get()	returns cue as a dict
cue.get_json()	returns cue as a JSON string
cue.show()	prints cue as JSON

---

• Full Example

from threefive import Cue
b64 = "/DBIAAAAAAAA///wBQb+ek2ItgAyAhdDVUVJSAAAGH+fCAgAAAAALMvDRBEAAAIXQ1VFSUgAABl/nwgIAAAAACyk26AQAACZcuND"
cue.decode(b64)
cue_data = cue.get()

---

Stream Class

class Stream(builtins.object)
 |  Stream class for parsing MPEG-TS data.
 |  
 |  Methods defined here:
 |  
 |  __init__(self, tsdata, show_null=True)
 |      tsdata is an open file handle
 |      set show_null=False to exclude Splice Nulls
 |      
 |      Use like...
 |      
 |      from threefive import Stream
 |      
 |      with open("vid.ts",'rb') as tsdata:
 |          strm = Stream(tsdata,show_null=False)
 |          strm.decode()
 |  
 |  __repr__(self)
 |  
 |  decode(self, func=show_cue)
 |      Stream.decode reads self.tsdata to find SCTE35 packets.
 |      func can be set to a custom function that accepts
 |      a threefive.Cue instance as it's only argument.
 |  
 |  decode_fu(self, func=show_cue)
 |      Stream.decode_fu decodes
 |      1000 packets at a time.
 |  
 |  decode_next(self)
 |      Stream.decode_next returns the next
 |      SCTE35 cue as a threefive.Cue instance.
 |  
 |  decode_program(self, the_program, func=show_cue)
 |      Stream.decode_program limits SCTE35 parsing
 |      to a specific MPEGTS program.
 |  
  decode_proxy(self, ffunc=show_cue_stderr)
 |      Stream.decode_proxy writes all ts packets are written to stdout
 |      for piping into another program like mplayer.
 |      SCTE-35 cues are printed to stderr.
 |  
 |  show(self)
 |      displays streams that will be
 |      parsed for SCTE-35.
 |  
 |  ----------------------------------------------------------------------

---

 threefive.Stream(tsdata, show_null = False)

• src stream.py
• The threefive.Stream class parses SCTE35 messages from a file or stream.
• Supports
  • Multiple Programs.
  • Multiple SCTE35 Streams.
  • Multi-Packet PAT, PMT, and SCTE35 tables.
  • Constant Data Parsing.
    • threefive.Stream is designed to run continuously

Method	Description
Stream.show()	Prints Streams that will be checked for SCTE35
Stream.decode(func=show_cue)	Prints SCTE-35 cues for SCTE-35 packets. Accepts an optional function, func, as arg.
Stream.decode_next()	Returns the next SCTE35 cue as a threefive.Cue instance.
Stream.decode_program(the_program=None, func=show_cue)	Same as Stream.decode except only packets where program == the_program
Stream.decode_proxy(func=show_cue)	Same as Stream.decode except raw packets are written to stdout for piping to another program.

Stream.show()

• List programs and streams that will be checked for SCTE35 data.

>>>> from threefive import Stream, version
>>>> version()
'2.2.69'
>>>> with open('video.ts','rb') as tsdata:
....     strm = Stream(tsdata)
....     strm.show()
....     

Program:1030
        PID: 1034(0x40a) Type: 0x6
        PID: 1035(0x40b) Type: 0x86 SCTE35

Program:1100
        PID: 1104(0x450) Type: 0x6
        PID: 1105(0x451) Type: 0x86 SCTE35

Program:1080
        PID: 1084(0x43c) Type: 0x6

---

Stream.decode(func=show_cue)

import sys
from threefive import Stream

if __name__ =='__main__':
   with open(sys.argv[1],'rb') as tsdata:
       sp = Stream(tsdata)
       sp.decode()

• Pass in custom function

• func should match the interface func(cue)

import sys
import threefive

def display(cue):
   print(f'\033[92m{cue.packet_data}\033[00m')
   print(f'{cue.command.name}')

def do():
   with open(sys.argv[1],'rb') as tsdata:
    sp = threefive.Stream(tsdata)
    sp.decode(func = display)       

if __name__ == '__main__':
    do()

---

Stream.decode_next()

• Stream.decode_next returns the next SCTE35 cue as a threefive.Cue instance.

import sys
import threefive

def do():
    arg = sys.argv[1]
    with open(arg,'rb') as tsdata:
        st = threefive.Stream(tsdata)
        while True:
            cue = st.decode_next()
            if not cue:
                return False
            if cue:
                cue.show()

if __name__ == "__main__":
    do()

---

Stream.decode_program(the_program, func = show_cue)

• Use Stream.decode_program() instead of Stream.decode() to decode SCTE-35 from packets where program == the_program

import threefive

with open('../35.ts','rb') as tsdata:
    threefive.Stream(tsdata).decode_program(1)

---

Stream.decode_proxy(func = show_cue)

• Writes all packets to sys.stdout.

• Writes scte35 data to sys.stderr.

import threefive

with open('vid.ts','rb') as tsdata:
    sp = threefive.Stream(tsdata)
    sp.proxy_decode()

• Pipe to mplayer

$ python3 proxy.py | mplayer -

---

Issues and Bugs and Feature Requests

---

Speak up. I want to hear what you have to say.

If threefive doesn't work as expected,

or if you find a bug ,

or if you have feature request,

please open an issue.

If you want help resolving a video parsing issue, a sample of the video is required .

Jimmy?