Description as a Tweet:

SteadySound allows for hands-free control of a device’s volume, where the softer sections of video will be made louder for clear audio and the louder sections will be made quieter for less jarring audio. This allows for a more overall equalized listening experience. #SteadySound

Inspiration:

In this generation, a common daily activity is viewing videos on media devices. This could be the morning news, an educational video, a class, or even just pure entertainment. Typically, people find themselves in locations where they may want to share audio with other people. For example, watching a movie or a sports game would require the use of external speakers for everyone to hear. Since different editors create their own content, sound is often inconsistent between videos. As a result, multimedia users often find themselves constantly scrambling for the TV remote or other volume controls when the content is too quiet, or disturbingly loud for others in the vicinity. This team recognized this inconvenience and sought to offer a solution. The resulting product was SteadySound.

What it does:

- SteadySound is a device meant to reduce the range of volume that occurs between the scenes of a movie or video. If there are sections of a video that are too quiet to be heard, SteadySound will increase the volume in order to make the video more audible. Conversely, if a section of video becomes too loud, SteadySound will decrease the volume in order to achieve a comfortable volume as specified by the user.

- We built it to work as a standalone media server, which can be set, and forgotten, on a media device like a TV. The server holds movies, plays them, and allows for the control of their playing, and their normalization for better movie / video watching.

How we built it:

- The backend of the website is built on VueJS and Babel. The dashboard is a single-page application. On the server-side of the web component is a series of python scripts that process uploaded videos. One python script monitors the upload folder and takes new movies, creating a normalized version of their audio using ffmpeg-normalize. The other python scripts have to do with the operation of the media player.

- The media system is made up of three two parts, a master server and a client server. The master server opens up a Unix Domain Socket and waits for clients. The clients receive queued commands from the master server which dictate how their video files are played.

- The python master server is connected to the web front-end with a websockets connection. This allows for the web front end to communicate media commands to the media player, which runs on qt5 and libVLC. The bidirectional nature of the websockets connection means that the web server and the python back end can communicate the whole time.

- Video files, when they’re uploaded, are indexed in a local sqlite db.

- We also used Unity + Vuforia with C# scripting to enable the VR component and IP searching.

Technologies we used:

• HTML/CSS
• Javascript
• Node.js
• Vue
• SQL
• C/C++/C#
• Python
• Raspberry Pi
• Other Hardware

Challenges we ran into:

- Splitting the project into smaller deliverables that could cater to each team member’s unique skill set while still maximizing the efficiency of the collective team.

- Developing a method to identify a device’s IP address using a physical MAC address.

- Maintaining communication between group members in terms of what each component in the project was used for and creating documentation based on input.

- There was a massive challenge both when it came to uploading the video and managing things in the database with Vue, i’m new to using Vue and npm to host websites, and so when it came time to be querying data, it was very difficult
Because we used a raspberry pi there were some limitations to the packages that we had access to, for instance it was quite the process to get qt5 working on the raspberry pi. Also we had some issues with our audio drivers that required us to fiddle with the low level configuration libvlc.

- We couldn’t get the raspberry pi connected to eduroam so we had to emulate a home intranet with a cheap router we picked up from walmart, this allows us to demonstrate what the server would be like wheh used in a home, although having to work off of multiple wireless networks while developing the product was rather difficult

- The biggest failure that we encountered in the project was the interaction of Vue, Webpackage, Babel, and websockets. We tried to use Unix Sockets initially for the communication between the javascript server and the main media server, but that doesn’t run in the browser, so we had to resort to websockets. This meant we needed the master server to host a connection both on unix sockets but also as a websockets server. This was difficulty to arrange. And pretty much every single package that we tried to get websockets to interact with VueJS called an error, we even tried it on a device other than the raspberry pi, to consider having the webclient being separate from the headless media player. It is possible that we could have arranged it in such a way that the website is local and manages the raspberry pi and its media server alone, but this wasn’t possible because, once again, websockets, in every package that we found that used it, wasn’t compatible with our node environment.

Accomplishments we're proud of:

- Working with Unity and Vuforia is a newer experience for us. It was really fun to pick up on new skills and practice working with software that we have barely touched in the past and come out of it with a working product for AR.

- I’m particularly proud that we got at least part of the front end for the website working, with at least a progress bar that displays increments in terms of movie time. I also think that building the media player in qt5 is quite impressive, and in particular, using a master server to communicate with another media player using Unix Sockets for asynchronous commands to the elements of the server.

What we've learned:

- How to better communicate as a team.
- Project management, how to utilize each person’s skills while maximizing efficiency in order to meet our goals in terms of deadline times.
- We gained first-time experience with C#.
- We picked up on new experiences and skills regarding the use of augmented reality.
- Inter-program async communication with sockets.
- UI building with qt5 and GTK.
- How to use Vue to create simple web apps (but now how to get websockets to work).

What's next:

We would like to develop this into software that could theoretically run in realtime on any device. For example, while watching YouTube with earbuds in / headphones on, switching to a new video can be jarring (and damaging) if the video is significantly louder than the previous one (especially if you had the volume all the way up to compensate for a quieter video, such as ASMR).

Built with:

- Augmented Reality
- Raspberry Pi
- Unity
- Vuforia
- Javascript
- Python
- C#
- HTML
- Node.JS
- Vue
- SQL
- SolidWorks/3D Printing
- Ffmpeg-normalize
- libvlc

Prizes we're going for:

• Best Software Hack
• Best Web Hack
• Best Venture Pitch
• Best Hardware Hack
• Best Beginner Software Hack
• Best Beginner Hardware
• Best Mobile Hack
• Best Domain Name from Domain.com
• Best Hardware Hack

Jeffrey Matheson
Vineeth Rao
Thomas Morton
Hasan Khwaja

Table 43 (TBL)