Inspired by the success of neural transfers on art the first thing that crossed my mind, was this is superb! Now er are going to make AI learn music. Can it be made t learn that? I learnt harmonium when I was small but could not find time to learn the flute. Well, if I can't learn it there is no issue, I could also make AI learn it and play all my favoruite songs. All it requires is a little tuning with librosa
How can one make it follow the tune? what are the parameters to be learnt? What to apply descent on?
Its the spectrum that needs to be learnt, how can that be done? Its simple we need to convert this one dmensional sequence to a 2d one, like an image and then apply neural transfer techniques
How can that be done? USing fourier transforms to ocnvert a 1d seignal to 2d one and then apply convolutional neural networks
Here is the sherlock theme song (copyrights with BBC) playing and AI in the background trying to learn to play flute using tensorflow. The computing power provided was lower and even on GPU training for 2 days, we could not make deep learning crystal clear. So the best thing it captures is an essence of the song, with little pauses in the middle and a follow along tune to accompany!
Librosa can read those spectrums and generate tuning plots for the songs and the music style to be transferred, our job is simply to minimise errors generated and make the neural network learn, by passing the original sound through filters. The net is a relu activated 2d convolution neural network where loss is minimised using weights attached to both the content where the style has to be transferred and the style that has to be learnt, in this case a flute. The weighted sample can then be normalized with the number of samples. Playing around with weights can help in better tuning and adding more filters may help increase its audio quality too.
What are the next steps? To make my favourite singers sing my lyrics? How can that be done usinga NLP generator to write new lyrics for me and then using style transfer to transfer my singer's singing style!
I also did some experiments with brushstrokes on how code can be used to do ray tracing and generatea beautiful potrarit. Taking help from an online repository, I modified it to change the angle of holding the brush, pressure strokes, etc. But then the thought crossed my mind, what if I made it to paint in colour?
It is just rays, I can make it paint whatever colour I want by simply changing the ray's color! It generates great pictures like this!
