Portfolio of Toni Sanio

An app that introduces brainwave entrainment for various use cases in form of premade presets and editor that let's user make own sesssions and import or export them.

A platformer game with brainwave entrainemnt to have focused flow experience and relaxing gaming using graphics taken directly from nature.

A platformer that tries to immerse player into focused still relaxing and still challenging levels with psychedelic twist.

Luckily we have artists (Zephyder Group) with passion but we have hard time finding more programmersr and level desingers so might end up doing some procedural solution at some point.

A game we put together with my brother Teemu (Zephyder Group) in order to learn how to release a game in the Google Play Store and gain generic game development exprience.

My work was to manage project, design it partially, do programming (Unity, mobile optimization,..) and some harsh 3D models and other graphics. The action packed audio atmoshpere with unique electronic music and sounds are the result of Teemu working in his studio. Besides audio Teemu did also some 3D graphics like the player character and brilliant design work.

The game is available here in the Google Play store.

This was meaningful for us in a sense that it was the first of the many projects we have done that actually got published somewhere where people can reach it. After this we kept our motivation and learned couple of things on how to run a project from start to an end. And at the moment of writing this we are already working on a bit bigger game title with more people bringing more special talents together in Zephyder group.

This year theme was about waves. So everyone had to make game based on that theme. Usually I work in group but this year I felt like why not just make once independently. I had some idea about game based on sounds in my subconcious mind before the event so this was interesting place to realize some small prototype on some part of the idea.

All these orbs make sine tone of different frequency. When they are combined in the game they make quite interesting sound effects and binaural beats. The point was that player would have to listen what kind of sound each door makes and what sounds the balls make or what sound they make when they are combined and then player would have to open doors using that correct frequency. I had to add some colors etc to make it more understandable now. It looks like the idea might be worthwile to continue working on later.

This project consisted of programming 2.5D character controller, other physics related functionalities, character animation, randomized cognitive puzzles etc.

Picture of the table while running the game

We were in Scifest 2016 with Luonnollinen Koulu. We had many kinds of examples about the games we were developing those times. One of the games was this card parsing game that I made using Java, LibGDX and TUIO-protocol.

The game was running on multitouch table prototype that was built by Harri Karhu earlier. Basically hardware consisted of video projector, IR light sources and high frame rate IR passing filtered camera. TUIO-protocol was used to handle the multitouch input for the actual game.

The game was running quite smoothly but sometimes sun light was a bit problematic because of the massive infrared sun emitted. That interfered with the table's infrared based recognition of touches. During evenings the table ran the game fluently and made quite illuminated atmosphere in our workshop.

Basically this started as a small game 2013 that was more like just game where player floated in the space and picked up something.. Filled the suns with some energy and completed levels meanwhile avoiding enemies.

Later I wanted add there some planets.. With radial gravity!.. Like really round planets!.. Destructible terrain.. Multiplayer..

I wanted the universe to have solar systems where player can fly from planet to planet. Planets rotate around sun.. Unlimited area for player to explore.. Limitlessly generated world!

..and liquid! And oh boy did it took long time to study and epxeriment with liquid simulation. Two years or so. I couldn't find already made solution that was good enough for the game so I had to do one. I was experimenting with many kinds of solutions like just creating polygons or pools for water but they didn't behave very well.. SPH, hybrid SPH that merges and splits particles, other optimization of SPH like doing calculations on GPU. Then I also did eulerian based solution completely using shaders to calculate things in GPU. Now the solution is closer to SPH & eulerian combintion and I try to post blog about it after some prototype of the game is ready.

The development of this game was smashed into bitbucket update mostly when we forgot to change from mercurial to git and few yeears code lost so might take a while to wrap it up...

If you haven't ever heard of it, check what neurofeedback is.

After working with my earlier Brain-Computer Interface software that was made in C++ I felt the development environment too complex and ended up making similar software in Java. After few weeks of development I'm starting to feel pleased about this path. This version seems much more easier to modify for various kinds of sessions. Also support for multiple kind of devices was something I was able to take into account before starting to program.

Currently the project code is in my personal repository but I will push it to GitHub in case some people are interested doing something with it.

Screenshot of the application 12/2016.

Idea of the app is to collect points submitting discounted products. User can also use the app to find discounted products from local shops. Users can take pictures of discounted food products and earn points and later these points can be used to get rewards like coffee or discounts in many places.

User can install the app on any other phone with Google Account and it should still use the same account.

I wanted to participate helping guys make 3D space shooter. It needed optimized collision detection for rays and collisions between objects and retrieval of objects in space. So I made OcTree. It reminded me so much QuadTree so it was fascinating to implement. It seems to work quite well. The demonstration video shows performance difference between OcTree and just brute force.

Sourcecode for the project: https://github.com/tsanio/OcTree

During this time I was helping to create user interface for an eye tracking system. In which the users could create tests and see results. The tests could have been ie pictures or multiple pictures representing the same thing with different arrangement (A/B-testing). After user had posted the tests, testers could find them and run them using eye tracker. And after that user could see that testers had done the tests and inspect them using heat map to see where the testers had been looking. The user interface consisted of many elements like login system, browsing of results, inspecting results with various tools like heatmap or selecting areas and inspecting percentages of gazing during specific time period.

The used tools and platforms to create the interface were mostly Javascripts, Less/CSS, AngularJS, jQuery, HTML5 (Canvas element related), and the API other guys were creating in the backside.

Just picture to give the idea of heatmap:)

Abstract
Topic of my thesis was to research possibilities of brain- computer interfaces as meditation aid. To do this I looked into meditation history and it's basic methods which provided the basic philosophy and guidelines for building modern tool to aid meditation. Then I studied and wrote about the phenomenon that are seen or measured during meditation state or as a result of long tem meditation practice in brain. The electrical activity of brain related to meditation would provide an possible interface for measuring meditation state or help reaching it with means of brain-computer interface.

I also did list of different ways to alter state of mind externally via technology. These included transcranial magnetic stimulation, biofeedback, neurofeedback and brainwave entrainment which seemed very promising as a result. From these methods I chose to combine neurofeedback and brainwave entrainment. I then constructed a system which guides individual meditation session with audio feedback. The audio consisted mostly of nature sounds for example sounds of ocean waves or fountain. The given audio feedback reacted to person's EEG and tried to encourage person to change his state of brain into more meditative state. New things in this study was for example non-medical hobbyist EEG-device and changing audio environment to prevent boredom.

In the end I did an experiment to verify the operability of the system with 16 participants with two different sessiontypes. It was possible to conclude that two different groups could be distinguished from each other by their avarage EEG recordings. From the session comments written and given by participants it was noted that they experienced relaxation, decrease of anxiety, hypnagogia, coming up of memories, silencing of inner dialogue and changes in thought patterns.

Meditation
Meditation is a practice to improve mental health with ancient origins. In thesis I took look at the origins of meditation far as written material could provide. This included the Nubians/Egyptians which were the oldest. After that I described the basic principles of Patanjali's method which seemed to be the basis for most of the modern meditation methods. About modern methods I took deeper look into Vipassana which involves concentration and mindfulness training. These were used as a basis for building the system.

Ancient asana in hieroglyph.

EEG of meditative brain
Meditation gives different benefits for person who is practicing it: lowered blood pressure, decreased anxiety and overall improved mental health. These benefits comes with effects that can be measured from person's brain structure or brain activity also. Firstly It causes trait effects which can be measured as a result of long term practice of specific meditation styles. Secondly it causes state effects which can be measured during the actual meditative state. In thesis I mostly concentrated on the EEG-activity rather than physical changes. These EEG state effects include decreasing the overall amplitude of all brainwave bands from delta to gamma, increase in theta, gamma or alfa amplitudes, decrease in beta amplitude.

Ways to help changing brain state
There exists many ways to alter human state of mind. This has been done for example by our ancestors during shamanic drumming. The beat of the drum helps brain to achieve deeper state via following the rhythm of the shaman's drum. Also other kinds of methods to help meditation had been used ie. mandalas, mantras, .. Here I gathered modern methods that can be used via PC. These included ie. transcranial magnetic stimulation, biofeedback, neurofeedback, brainwave entrainment. I noted that in field of Biofeedback there has been this interesting experiment which helped participant to reach mindful state by hearing sounds which were linked to person's own breathing. In this way the person could be more mindful and explore the audio environment without getting trapped feeling guilty of loosing attention. I also discovered some interesting results from neurofeedback and brainwave entrainment.

Building the system
The system was built from EEG-SMT-device as basis for the hardware EEG-device and C++ as basis for the software. The idea of the system was to play audio to help person reach more meditative states. This was done similar way as neurofeedback: Getting positive feedback from meditative brainwaves and negative feedback from non-meditative brainwaves. The audio consisted of nature sounds and brainwave entrainment. The volume of the audio was changed according user's brainwaves and the user also got prize sounds after reaching high amplitudes. When user reached enough of prizes the audio environment was changed. This was done in purpose to prevent dullness which is one factor when considering practicing mindfulness. Two session types were created: gamma-amplitude increasing session and theta/alfa- session which was meant to increase theta and alfa amplitude but mostly theta.

Shield made for EEG-SMT by Harri Karhu.

Screenshot from the application.

Testing the system
Testing of the system was made with 16 participants. Participants sit during the session and listened the feedback given from their brain activity. The results suggested that both of the groups could increase the typical brainwave activity that was supposed to be trained during the sessions.

Particpant before the neurofeedback session.

Comparing different groups' results from baseline to session avarages. Values are relative to other current amplitudes.

Source code for the application here. Harsh version. Feel free to use it for any purpose

The idea of this project was to make music player which detect what kind of mood the user has and play that kind of music. Or user could for example pick choice like "I want to listen music that makes me active and happy". The program also learned and updated the reaction of users mind after each time listening part of song or complete song. So it could forexample just switch from song part that user reacts to certain way and then change another one where user reacts quite similarly.

Project had also complete website. The program code is in GitHub If you are interested. For me this was very fascinating and entertaining experiment. I'm looking forward making this in Java as well again.

Screenshot

This project I programmed the Windows Phone version of the client app. The application read routes that i.e. teachers had made for students. Students could navigate using the app like a compass through the route. The routes contained locations where user could find i.e. videos or pictures or so for example to learn something from history of the location. Project homesite.

This was a summertime project job for Science Park in Joensuu at 2013 summer. The program mimics decoration program where user can place objects and elements into room. The main thing here is that all inserted things in the room affect the atmosphere of the room. This climate of the room is seen in the meter and user can do trials how to make the room spoiled or nicely decorated for good air environment. Also it was made possible to record "game play" to easier demonstrate something later again for someone.

This was a Bachelor's thesis project I made for university. I was curious about what to do with EEG reading devices. I found this interesting idea that you could adjust the challenge of the game to make it more interesting according the player's mental state. What does this mean basically is that If the device can read from player whether he is alert or relaxed, focused or not focused, we could alter the game play in order to push the player to stay in the sweet spot where difficulty of the game play is matched with the skills and mindset of the player properly.

The device I got home those times had already these values Attention and Meditation which kind of tried to provide something that I could use as a basis for relaxed-not relaxed and focused-not focused values. So I ended up using thsese values to define whether the game should be made more difficult or easier. The game brought more difficulty to the player by increasing amount of enemy soldiers on the fly. If player was having too difficult experience i.e. not calm and too focused, the spawn rate of soldiers was decreased.

This was kind of funny idea, but somewhat difficult to actually get something out of it. It might be worthwile to compare the player experiences with sham devices and also just some other kind of flow mechanic which is not based on eeg but the player's skill on the game. This would provide interesting data.

The actual game was made using Allegro and C++ earlier and modified later for this project accordingly to adapt to the user mind using MindWave EEG reading device.

My friend was studying how people pay their debts and was in need of software for that so I quickly put up software to run experiments.

Related paper published later Understanding Cross-National Differences in Risk Through a Localized Cultural Perspective

Source code and binaries.

In this game you will play against AI and try to avoid the ball hitting on your platform. You will get points by making the ball drop into opponents area.

Play the game here

And here is another game from the summer called Waamz. I was missing game from my childhood called boogalooper where player must catch much items as possible using rope without letting the rope touch the items. So the idea is to catch moving objects using mouse trail and avoid hitting the objects. The more objects at once, more points!

Play the game here

Pebl-test is a program which mimics the properties of a program called Pebl. Pebl stands for PEBL: The Psychology Experiment Building Language and can be found from http://pebl.sourceforge.net/. In pebl-test one can create own scripts, use already made scripts to run different tests. The idea of this little pebl-clone is to make this same test program available for open source java. At the moment there exists three different tests that can be run and they're very like the same as in original pebl experiment.

The idea is basicly that you can run some tests where you have to pick something from the screen and in the end there will be log-file that records the reaction time and those results could be used for example research purposes.

This was a project related to cognitive science. Documentation page is here and can be found in the source code package.

Traveling Salesman Problem is the problem where exists certain amount of points for example cities and the goal is to find the shortest route that visits each city and returns to the original place. It can be seen as a salesman's travel that tries to cut fuel consumption of his car and still visit each city. This is mainly why the name is like that.

I would describe the Traveling salesman problem as one of the most studied algorithmical topics in the world. This is because of it's very asked problem for example in the means of logistic. It's also NP-hard which means it's not that easily solved but instead it's computationally difficult problem.

Even prizes has been set for the one who resolves it better. Still no effective solution method is known for the general case. indeed, the resolution of the TSP would settle the P versus NP problem and fetch a $1,000,000 prize from the Clay Mathematics Institute.(TSP,2013) There also is some movie about the problem Travelling Salesman 2012 where four mathematicians are hired by the US government to solve the most powerful problem in computer science history..

Here I'm trying to resolve this problem in means of Tabu Search. There exists many research papers, articles and even whole books about Tabu Search but here I'm just interested on the basic stuff of the algorithm to make some kind of working solution. To understand the basics it's good idea to get grap of what Local Search is as Tabu Search is a sub category in local search. Local Search tries to find an initial route to the problem and after that it tries to find better solutions by changing the neighbours. In this way local search tries to find locally better resolutions and after that it continues to do the same thing with the new best result. This is also the case in Hill Climbing algorithm where the solution is always improved step by step. You could imagine yourself walking or jogging a path and every time changing randomly some way point during the path and in the end calculating how long was your path or how many walking steps did you have to make for example. The when you find better solution you will use that and try to improve it by random changes again. You only change one point of the path at once and return if it was not good.

The tabu search name comes from tabu: to denie. For the algorithm this means that it denies some options that are not allowed. For example in this scenario I'm interested to make the algorithm create a tabulist of changes that would cause the algorithm to jump back to old resolution and this would cause it not to go further so well. This is usually the case in tabu search as Jennifer Mosher [2] puts it in his page The Tabu Search uses memories, called Tabu Lists, to record the recent history of the search. This prevents the search from going back to previously visited solutions.

Sometimes the case is also opposite. When the algorithm is stuck there may be use to release it and let it perform a tabu movement. The previously mentioned article also says that sometimes tabus can be too powerful, by stopping steps that would not have caused cycling and even making the search unable to go on. This is why Glover also has aspiration criteria that allows you to disable a tabu. In theory, a tabu search would go on forever. This is why there are three most commonly used termination criteria. In other words we are interested to either allow tabu, create some other alternative solution when local optimum is reached or just plainly stop the process after certain amount of iterations to stop the unfruitfull searching.

This algorithm

Here my algorithm was purposefully made to read a matrix that contains the distance matrix of the possible routes between each places or nodes. This case also was a bit different from normal tsp that it had starting and ending point rather than cycle. It actually didn't change much the algorithm, just had to take that into account anyway. My implementation had a demo-propertie that loaded already defined distance matrices and results from other implementations that had calculated the distances already. Then it tried to calculate a resolution using Tabu Search and then comparing it to the other results to see If it was close to those results. I also implemented there a kind of possibility to generate randomized maps and calculate there routes. This is also good for testing purposes. I used just plain Java to do this.

My implementation here is based on the idea to swap each of the path nodes that are not in the tabulist and keep the best solution. After that write down the new tabu swap and do the previous process again to the new best solution. (Swapping means to take one node and change it's place with another node on the path). Here the actual method in simple pseudocode is to:

During this implementation I tried many different ways to get better results: I tried varying the swap node amount. For example I tried to swap three different nodes at the same time. I also tried to gather second best solution that would keep track of the second best solution and that would have been a kind of alternative route to find new paths if the primary way was stuck in local optima. BUT this seemed to either follow the primary way and lead to same resolution or it never achieved better results. I tried brutal ways like randomizing/shaking totally the ordering(no luck with this.. nada..), allowing tabus(okay it returned to old.. what's the fun here? See it to repeating?), having a brutal index for each swap and proceeding those if no imporevement was found in one trial. Also I had recognition if there wasn't primary improvement the algorithm would go to improve the second but.. As I said the second way wouldn't succeed any better. I also noted that this swapping through different gaps seemed to be the same as swapping each neighbourhood- This just seems like another view point to it.

The Sourcecode

The Source Code consists of files:

Downloads

DAA project 3_7_2014 no slides

Ask for other versions If neccessary..

Running the program

There is more info about how to run the program in the TabuSearch.java, but you may not want to look into that If you just need to run this algorithm with some input distance matrix. Basicly to run this program you just need to change the distance matrix that is in the input.txt and the output.txt will the the result after the program is finished. Note that the last node in distance matrix is exception in a way that IF it's set to zero in all points it means that the node is not taken into account at all(filtered off). This is because the user location can be given in the last point and sometimes if it's not delivered we can take it off and just use the actual points. To run the program you have to either run the jar which is the simplest thing to do or run the application with java-command. To use the jar just simply run it as a normal application in your OS. So in command line you can just write:

C:>TabuSearch\TabuSearch.jar

and it will run the program with given input and print the output to output.txt.


You can also give the starting vertex as input. C:>TabuSearch\TabuSearch.jar sX

where starting vertex will be given as X.

FOR MORE PARAMETERS SEE THE TabuSearch.java..

If you cannot compile it but need to run it then you can extract the classes from the jar by using it like

C:>TabuSearch\jar xf jar-file [archived- file(s)]
(See more in http://docs.oracle.com/javase/tutorial/deployment /jar/unpack.html)

If you want to compress all the stuff again write this:

C:>TabuSearch\jar cfm TabuSearch.jar manifest.txt *.class

The actual practical results in O-Mopsi

Demo results list

Demo results are excactly good as the offered results except one occasion: Demo 5. (distanceMatrix_SciFest_Hard) doesn't reach good resolution as the given solution. The resolution is:

And the given result in demo file is:

0.16.. difference is not much, but it's still something. This was something I couldn't figure out here to work better even tried many different things.

References

[1] TSP, site owned for TSP-problem (http://w ww.tsp.gatech.edu/problem/index.html, 2013)
[2] Search Algorithms, Jennifer Mosher(htt p://www.csun.edu/~jam79241/jennifermosher.html, 2013)


Thanks also to some other articles that I read while studying and developing this(I forgot the sources of those hmm..). Also thanks for this guy named voidException (his tsp- page) who had done a implementation on java for tabu search. I found this when I was just finishing my algorithm and it gave me good ideas how to clean it more. Also I edited voidException's code to work on the 5. demo data if it could had have better solution but it was worse. It's strange.. This was a good resolution for first when I looked into it- I edited this code to work without cycle and installed there the same graphs, but it didn't manage so well as this mine algorithm. I wonder why because it looked very similar implementation..?

Sometimes I make small utilities like this html+js portfolio webpage, alarm clock or for example this kind of 24 hour desktop clock in Java.

Shows time of the day in transparent clock on the desktop. Can be hidden in system tray and invoked again. New upcoming features should be *Themes. *Activity tracker to show when computer use is most active. Download JAR It's possible to add it into start up programs as any other application. Ie in Windows: WinKey + R -> shell:startup [enter] -> Copy the jar or shortcut.

Sometimes I work on small things on Arduino to make some automated things in home. It might include also modifying something on my small home server which runs in Linux where I also have to make Java program to control something. Some arduino projects are for example timered devices or motion detector based colored led lights. Here is one project

Ardunexa

Nexa remote sockets are like small sockets that can be attached between wall ac and some device ie. light in order to control it remotely. Normally it's controlled using the remote controller. It is possible to connect 3 devices per remote controller usually.

Here the idea is to make it possible to control the sockets remotely using Arduino and also to support huge amount of sockets instead of just three.

To accomplish this one needs to just do a bit soldering into the remote socket of Nexa or use some other way to accomplish the radio transmission. Then just install the library or write own code.

I had an opportunity to get guidance how to accomplish this task from Irah who originally reverse engineered the Nexa sockets. I made a easy to use library and description to support Irah's project which you can see and download here