Projects

Below are some projects that I have worked on during my studies, at internships, and over weekends at hackathons. Links are provided wherever available!

Research

• Data augmentation and room simulation for far-field recognition (2019-2020)
  As part of my work at Snips / Sonos, I worked on data augmentation and room simulation techniques for far-field recognition. Here is some of the work we did on closing the gap between simulated and measured room impulse responses.
  

• Microphone array processing (2015-2018)
  An ongoing project which was a part of my Research Scholar experience with LCAV. In collaboration with multiple people, most notably my supervisor Dr. Robin Scheibler, the following tools/data for microphone array processing were developed:
  • Hardware for the BeagleBone Black: cape for 8 PDM mics, CompactSix array, Pyramic
  • Software: browser-based interface, driver for 8 mic cape
  • Algorithms: pyroomacoustics
  • Data: Pyramic recordings
    
    

• Sampling at the rate of innovation for ultrasound imaging and localization (Master thesis, 2018)
  I investigated the application of Finite Rate of Innovation (FRI) and Euclidean Distance Matrices (EDM) can be applied to the task of ultrasound imaging and localization. The target was medical imaging but an application to non-destructive evaluation was also considered.

• Python implementation of DeepFool and applying it to commercial classifiers (Fall 2016)
  As a semester project, I implemented the DeepFool algorithm in Python and developed a “blackbox” approach, i.e. a situation in which we do not have full access to the classifier structure as is normally required by DeepFool. As well as applying this blackbox approach to academic networks, such as those submitted to the ILSVRC competition, I also applied it to a commercial classifier - Clarifai - to gauge the robustness of products in the area of image classification and object recognition. An implementation of DeepFool for Google’s Cloud Vision, IBM’s Visual Recognition, Amazon Rekognition, and Microsoft’s Computer Vision APIs was also made but not extensively tested due to limited query credits.

• Compressive sensing based ultrasound imaging demo (Spring 2016)
  As a semester project, I developed a MATLAB GUI and software for a demo of plane-wave medical ultrasound imaging based on compressive sensing techniques. This would interface with the FPGA-based ultrasound device ULA-OP. I was part of the team that presented the algorithm/platform at the 2016 Nano-Tera Meeting poster session.

Internships

• DSP Concepts (Masters internship, 2017)
  During a six-month internship at DSP Concepts, I investigated various beamforming strategies and developed a Beamformer Design Tool. With the tool, fixed beamforming weights in the frequency domain could be designed given a certain set of parameters such as the microphone array geometry, the microphone directivities, a desired source direction, a desired minimum beamwidth, and the expected signal and diffuse noise levels.

  
  Different beamformers could be compared and evaluated interactively over a variety of criteria such as SNR gain, polar patterns, beamwidth, and uncorrelated noise gain. Finally, the tool was linked with DSP Concept’s proprietary tool - AudioWeaver - in order to conveniently export the designed weights and/or design the beamformer(s) within AudioWeaver. A white paper which describes some of the beamforming and DOA strategies can be found here and another on voice user interfaces here.

• Fraunhofer IDMT (Bachelor internship and thesis, 2014 & 2015)
  As part of a three-month internship during my Bachelors, I worked at Fraunhofer IDMT in Ilmenau, Germany. There, I investigated and prototyped algorithms in Octave for GSM codec identification and (for a couple codecs) tampering detection. For my thesis, I ported the implementation to C++ so that it could be integrated with IDMT’s other codec identifiers. Finally, I used IDMT’s annotation and testing framework to evaluate the performance of the developed algorithms.

Hackathons

• Speak2Tex (jacobsHack 2018)
  Using a Snips Maker Kit and the Console, we built a voice assistant that can generate the corresponding LaTeX syntax for a wide range of functions and operations: e.g. polynomials, trigonometric functions, integrals, derivatives, matrix operations. We won Bloomberg’s Best Tech Prize and GitHub’s prize for best Git practices.

• A cognitive robot for helping identify motor disabilities in children (HackZurich 2017)
  We equipped ABB’s YuMi robot with a “mouth” and “ears” (laptop speaker and microphone + Microsoft Cognitive Services) and “eyes” (Logitech webcam + OpenCV) in order to automate a concise version of the M-ABC Test for diagnosing motor disabilities in children. We were second place out of around 150 teams.

• Food Cycle: online platform to reduce food waste (Foodhack 2017)
  We developed an online B2B marketplace for food suppliers to connect with entities (such as restaurants and canteens) to sell excess products at a lower price rather than resorting to disposal. We won the Food Save Challenge.

• DoctorBot: a chatbot for medical advice (HackZurich 2016)
  A Facebook chatbot that provides medical advice and can suggest a nearby doctor. Implemented in Python with several APIs (Infermedica, Google Static Maps, API.ai). We were among the top 25 out of around 150 teams.