Call for round the year submissions for Rootconf in 2020

Submit a proposal at any time in the year on DevOps, infrastructure security, cloud, and distributed systems. We will find you a suitable opportunity to share your work.

Framework For Lossless Data Compression

Submitted by Manas Malik (@manasmalik17) on Monday, 6 May 2019

Status: Awaiting details


A lot has been done in the field of data compression, yet we don’t have a proper application for compressing daily usage files. There are appropriate and very specific tools online that provide files to be compressed and saved, but the content we use for streaming our videos, be it a Netflix video or a gaming theater play, data consumed is beyond the calculation of a user. Back-end developers know all about it and as developers we have acknowledged it but not yet achieved it in providing on an ease level. Since the user would not never be concerned about compression, developers can always take initiative while building the application to provide accessibility with compression before-hand. We have decided to create a framework that will provide all the functionality needed for a developer to add this feature. Making use of the python language this process can work. I’m a big fan of Python, mostly because it has a vibrant developer community that has helped produce an unparalleled collection of libraries that enable one to add features to applications quickly. The Python zlib library provides a Python interface to the zlib C library, which is a higher-level abstraction for the DEFLATE lossless compression algorithm, we have a lot to do including the audio, video and subtitles of the file. We also make use of the fabulous ffmpy library. ffmpy is a Python library that provides access to the ffmpeg command line utility. ffmpeg is a command-line application that can perform several different kinds of transformations on video files, including video compression, which is the most commonly requested feature of ffmpeg. Frame rate and audio synchronization are few other parameters to look closely. This is an ongoing project and there remains few implementation aspects, data compression remains a concern when touched upon the design. We along with python community intend to solve this issue.


This talk includes the study of different compression techniques that are currently in use. Making use of all the existing libraries and theoretical concepts, we intent to create a new framework for developers, this will be the first in Python language that would provide all the necessary features for compression. Most importantly video files that consume data of use on daily basis. Handling each frame of the video is a very efficient and lossless process, so far it does not has any fast providing application. The field additionally demands collaboration of other tools outside compression tools existing. As Python is becoming more handy to developers, it is very
convenient for the developers to grasp algorithms and develop in the language. All the theoretical concepts are considered based on the study emerged so far, practical libraries used are very specific and not quite efficient. The framework includes all the codes and algorithms needed for the lossless compression. Python libraries that are already existing and making work with frames and video rates in the
background can increased in terms of the efficiency. There is a splitting into multiple patches of the images involved in the process of compression. For each of the patch, location (recurrent) of the pixel is found. Prior to the pixel value, the estimated locations of each pixel is placed, i.e. for the entire image. We make us of the LSB algorithm to perform the embedding process, after each frame has been compressed. MSE, PSNR, SSIM, are few of the metrics that will determine the superiority and the relevance of the framework proposed.


They just have to come. :)

Speaker bio

We presented our idea of Data Compression at PyCon Cleveland 2019, we received a great appreciation from the members of the community. We have been working on a python project for a while, it is now towards the end of completion. It’s a project that helps refugees and organizations that helps minorities and immigrants to settle for a living. I have tried best to make the interface of my refugee data camp project minimized. It has the potential of a generic product. Me and my team are currently working on a framework for lossless data compression using python, I have published an international research paper on the theoretical aspect of the data compression, we are looking the implementation part of it. I have always been a fanatic of python and love playing with the language. I am now also looking forward to new projects and have been researching on some open source projects like warehouse and zulip. It has now come to the mental state that I want to explore more and more, I have worked on GUI apps using tkinter. I have also worked on many machine learning libraries involving numpy, pandas, scikit-learn, and other tools and projects. Contribution and future of the language are now what fascinates me. I have further plans with python and would love to share ideas that I am carrying. Interaction with python users and developers will help me a lot with my research and projects, not just me the whole idea of collaborating with people will help society, products that might land up in future with interactions. Talk is going to be great catalyst between my current state and my goals. Talk will help me collaborate with other users and developers. I have been following talks on YouTube for a long time and I have received quite a good impression of the projects demonstrated by community members. I too have many ideas that I would like to share and contribute with the community. Lot of experience and interaction is been collectively shared at the conference that can help me and my team with our research.




  • Abhishek Balaji (@booleanbalaji) a year ago

    Hi Manas,

    This proposal looks interesting from an infrastructure and distributed systems point of view and hence I’ve moved your proposal to be evaluated under Rootconf 2019 ( To proceed with the evaluation, we’d need to see detailed slides and a preview video.

    Your slides must cover the following:

    1. Problem statement/context, which the audience can relate to and understand. The problem statement has to be a problem (based on this context) that can be generalized for all.
    2. What were the tools/frameworks available in the market to solve this problem? How did you evaluate these, and what metrics did you use for the evaluation? Why did you decide to build your own framework?
    3. Why did you pick the option that you did?
    4. Explain how the situation was before the solution you picked/built and how it changed after implementing the solution you picked and built? Show before-after scenario comparisons & metrics.
    5. What compromises/trade-offs did you have to make in this process?
    6. What are the privacy, regulatory and ethical considerations when building this solution?
    7. What is the one takeaway that you want participants to go back with at the end of this talk? What is it that participants should learn/be cautious about when solving similar problems?

    We need the updated slides by 21 May, to evaluate your proposal. If we do not receive any update, we’d be moving the proposal for evaluation under a future event.

  • Zainab Bawa (@zainabbawa) 11 months ago

    Hello Manas, moving your proposal to Rootconf round the year submissions to put it into consideration for future editions of Rootconf in Pune/Hyderabad/Delhi.

  • Zainab Bawa (@zainabbawa) 11 months ago

    We’ll carry out a review shortly.

Login to leave a comment