PyCon Pune 2018

With the 21st century drifting towards Virtual Reality and Augmented Reality, the need for 3D modeling is increasing day by day. From automating simple tasks to performing intricate surgeries, Robotics has spread its wings to every nook and corner of our lives. In such a scenario, it becomes essential to make the computers understand about the third dimension.
The talk is based on the development of software for 3D Scanner using Python and other technologies, the project-3D Scanner focuses on scanning an object and developing its three-dimensional image in a polygon format file which may be later used for a variety of purposes.The detailed source code of the software developed for the project has been published on GitHub ( https://github.com/animeshsrivastava24/3D-SCANNER-IITB/ ). Python and OpenCV have been used to write the software. Open source Arduino Board has been used to control the hardware.

3D Scanner is aimed for use in the fields of Education, Research and Medicine primarily. Since the project, the references and the resources used in the project are Open Sources, it is completely free to use, modify and develop the code according to the user’s needs.The talk also focuses on the application of 3D Scanner in the above fields.

The hardware setup is only required to capture a set of images from different angles of the object of interest whose 3D model we want to generate.In the talk we don’t require the hardware setup, we will proceed with a set of pre-captured images the details of capturing images is mentioned below-
Two lasers have been deployed in the setup. At the center, there is a platform where the object has to be kept for the purpose of scanning. The platform is rotated in steps by a stepper motor controlled by the software through a microcontroller.A line laser is used to set the reference axis for the camera and the other line laser is projected on the object in such a way that the beam takes the outline of the edge of the object.The camera then captures the images with the rotation of stepper motor at different angles.
The talk will focus on the software development-
The code extracts the contours of the laser beam and saves the coordinates for all the points in that contour and another piece of code integrates all the points into a single 3D image. For the convenience of the user, a complete GUI application has been developed, talk will focus on development on GUI Software also. Although there are many technologies present for 3D Scanning, most of them use expensive hardware and software. Since the technologies used in the project are completely free and Open Source, the only cost of the project is due to its hardware.
The various technologies used in the software development are-

1. Python 2.7
   Python was chosen as the default language for writing all the programs throughout the
   project. The version used by the developers was 2.7.6

2. PyOpenGL 3.1.0
   PyOpenGL is the most common cross platform Python binding to OpenGL and related APIs.
   It has been used in the project for showing the 3D graphics in the Tkinter window.

3. OpenCV (Python) 3.2.0
   OpenCV (Open Source Computer Vision Library) is released under a BSD license and hence
   it’s free for both academic and commercial use. Written in optimized C/C++, the library can
   take advantage of multi-core processing.
   This library was used for all the image processing done throughout the project. The
   capturing of the frames, extraction of contours, conversion of images to different formats,
   masking of images were done using the functions defined using this library.

4. Numpy 1.13.0
   NumPy is the fundamental package for scientific computing with Python. Besides its
   obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container
   of generic data.
   This library was extensively used throughout the project as the output format of the
   processed image at any step was in form of a numpy array. It was also used for calculation
   of final coordinates.

5. Matplotlib 2.0.2
   It was used for plotting the scatter plots and the surface plots.

6. Pygame 1.9.1
   This library was used to host the OpenGL window inside the Tkinter’s frame in the main
   window.

7. Tkinter
   Tkinter is Python’s de-facto standard GUI (Graphical User Interface) package. It was used
   for creating the windows and the widgets.

8. Pyserial 2.6
   This library was used for serial communications with the microcontroller from the code.

The project 3D Scanner was developed by me under Open Source Community and is free to be presented,used and modified according to the user’s choice.

Outline

The various outline steps required in Talk are described below:
The content of the slides are:
1.Overview
2.Review about the present technologies in use
3.Software Specifications
4.Hardware Specifications (Not required during the presentation)
5.Fault analysis and future scope of development
6.Documentation details

The objectives to be covered in the slides are:
1.Construction of depth from 2D images
2.Extraction of the useful data from the camera captured images by using Image Processing.
3.Code for all the algorithms designed and decided to be used
4.Hardware setup
5.User-friendly software interface
6.Plotting the output

We will proceed with a set of pre-captured images, the detail of capturing images will be conveyed via a video in (Step 1 of Talk)

1. Image Capture:
   A video demonstrating the hardware setup will be played.The video description-
   The set up requires two laser modules to be oriented in such a way that one helps to determine the reference of the axis (L1) and the other laser (L2) allows the camera to capture edges of the object (to be scanned) being intensified by it(L2). The laser (L2) is set at an angle of about 40-45 degrees with respect to L1. The laser (L2) throws a vertical beam of red light on the object and the camera detects it. Since the light falls on the object from an angle, it takes the outline of the object. This helps to extract the outer outline of the object from each frame. The setup is so designed to rotate a total of 360⁰ in 48 steps and capture 48 such frames in total.
   Hardware Setup:
   The hardware comprises of Arduino Uno, Lasers (Red, 650nm), Stepper Motor, Hardboard base and a platform for rotation. The Stepper Motor allows the platform in which the object to be scanned is placed to be rotated in desired steps and rpm. The laser setup facilitates frame capturing in required steps of rotation as explained above in this page.
   The next step of Talk will focus on software development procedure-the development of GUI,use of OpenCV,Numpy,Matplotlib,Pygame and Pyserial will be explained efficiently.

2. Extraction of contours formed by the incident laser:
   The frame detected by the image is then processed using OpenCV in order to extract only those contours (regions) which come under the incident Laser beam, leaving other parts of the frame totally black (R=0,G=0,B=0). The frames are numbered from 0 to 48 and saved as ‘.png’ files. But, the most important thing that we require from those processed frames is the coordinates of the points that make the contour. Hence, the coordinates are stored in ‘.txt’ files numbered from 0 to 48.

3. Conversion of 2D coordinates to Spherical coordinates:
   This conversion is required to get the value of Radius (‘R’), Theta (‘Θ’) and Phi (‘ɸ’). It is so because calculation of spherical coordinates of a point of the object is easy since we have the value of ɸ from the rotation of the stepper motor. The saved text files which contain the 2D coordinates of each frame are parsed to obtain X and Y coordinates of all the points associated with the contours and then used to calculate the values of (R, Θ, ɸ) of all the points collected during the 360⁰ rotation.

4. Formation of 3D object file (.ply):
   Although calculation of the coordinates in the spherical coordinate system is easy, but finally the 3D format file contains the coordinates of the points in the Cartesian coordinate system. Hence the calculated (R, Θ, ɸ) coordinates are again converted to (X, Y, Z) coordinates.
   The coordinates so obtained are used to write a 3D image file. Polygon File Format (‘.ply’) is used to store the final 3D image data. These files can be used in various 3D image visualizing applications like Blender, which is also an Open Source Software. Various other 3D files can be created, for instance .obj, .stl, etc.

5. Plotting the 3D image using Graphing Application:
   Either the final coordinates obtained can be used directly or the generated Polygon File Format (.ply) file can be used to display the 3D object. The function used for this purpose is Matplotlib Scatter plot (a plotting library of Python).The Matplotlib window allows user to pan, tilt and zoom the obtained 3D image.
   Another application 3DLoader.py has also been developed which directly opens the created 3D file and displays it in a window. An available library ‘Trimesh’ was used to accomplish this.

Requirements

For workshop, the participants may bring their laptops inorder to visualize and understand the software development process. Even if the participant don’t have a laptop,the session talk will focus on each part of the software development and code part for understanding.

Speaker bio

I am Animesh Srivastava, a 3rd year (pre-final year) Undergraduate student from National Institute of Technology Hamirpur,INDIA majoring in Electronics and Commnunication Engineering (www.nith.ac.in ).
I am an Open Source Contributor - FOSSASIA & FOSSEE IIT Bombay, a devoted person towards the field of Electronic and Computer Science, with interest in Embedded System, Open CV,and Web Development.
The goal of my life is to achieve high career growth through a continuous learning process and to keep myself dynamic, visionary and competitive with the changing scenario of the world.
Presently, I am looking forward for a Summer Internship Research Project for Summer 2018.
Linkedin: https://www.linkedin.com/in/animeshsrivastava/
Github: https://github.com/animeshsrivastava24/3D-SCANNER-IITB

Links

• Presentation Link: ( https://drive.google.com/file/d/0B5mJPI80WenvV29VbDNhNVRUVjg/view?usp=sharing )
• Software Screenshot Link: ( https://drive.google.com/drive/folders/0B5mJPI80WenvMXBia1RoSE8yaXM?usp=sharing )
• Video Link:(https://drive.google.com/file/d/0B5mJPI80WenvTjBreGJQVU8zY1k/view?usp=sharing)

Slides

https://drive.google.com/file/d/0B5mJPI80WenvV29VbDNhNVRUVjg/view?usp=sharing