Pitch to Midi
By Bernd Kohler, forwarded by Mark Pleatman
I thought my question and its answer might be of interest to the group:
From: Bernd Kohler <bekosoft@tibalt.supernet.ab.ca> To: brill@oakland.edu
Subject: Pitch to Midi
Hello Mr. Pleatman
On Thu, 5 Oct 1995, you wrote:
> I noticed your recently uploaded program for detecting the frequency of
> played music, and thought you might be able to answer a question I've had
> for a number of years. How difficult would it be to write a program to
> convert played music do midi?
In my humble opinion, conversion from pitch to midi is very difficult.
> In other works, could you record somebody playing the piano, and detect
> all the notes being played along with their amplitude and length, then
> convert the information to a midi file that could be edited and played
> back on a Disklavier or other such instrument?
There may be some hope for this specific instance, where only one instrument is involved. To do a good job of it, I don't believe that it would be possible in real time, however. Of course, if you were using a midi keyboard instead of a 'normal' piano, you can easily record the midi directly with a midi interface and a midi sequencer package. Some companies also manufacture other midi instruments, like midi guitars. There is even a company (DigiTech if I recall correctly), that has come up with a system for vocalists, using special sensor to do this.
> I am sure this would be fairly difficult, but in principal it ought to be
> possible. There would be interesting commercial possibilities, such as
> rejuvenating old 78 recordings made by famous artists; as well as playing
> a record through a real piano (if the transfer could be done in real-time).
In principal, it may indeed be possible, but the problems go much beyond the detection of a pitch that I have accomplished with my products, Perfect Pitch and In Tune. I can assure you that just to do as much as measure a pitch accurately to 1 Cent, has been very difficult. Perhaps you can relate to this simply by the fact, that there are very few programs on the market that can do this at all.
I will try an explain the problems with pitch to midi a little bit further. To measure even a single note of one instrument is difficult, because the instrument's sound is made up of many frequency components. For example, when you pluck the E0 string of a guitar, the sound will contain frequency components for notes E0,E1,B1,E2,G#2,B2,D3,E3 (fundamental + 2nd to 8th harmonic). Now most of the time, E0 and E1 will have the largest amplitude so that my software, which assumes that only one note is played, can determine the correct note for the string. If I couldn't assume that only one note was being played, the task becomes impossible, because notes B1 and E2 (3rd and 4th overtone for E0) also happen to coincide with the B1 and E2 strings of the guitar. So, how do you tell a string's harmonic apart from other strings ? The only way that this could be done, is if a complete frequency spectrum which includes all of the overtones for each string could be computed and analysed. This task would take considerably more computer time than what I can currently achieve with In Tune for example. Now, add to this the fact, that the spectrum for the guitar notes are not constant, but vary over time, vary with the selected pickup, with the position of where the string is plucked, and many more, and you can perhaps appretiate, that the task becomes almost impossible. To the best of my knowledge, you will see this if you look at the special midi guitar pickups that one can buy. They appear to be dealing with each string of the instrument separately, which makes the job easier again.
In some of my research and tests, I have found, that at this point in time, most of the computer's time is required simply to compute the spectrum for the recorded sound, and leaves no time to perform the complex analysis that I am hinting about in my previous paragraph. This is where the human ear has a distinct advantage over the computer. The ear's structure already relays spectral information to the brain, it doesn't record sound and then converts it into a spectrum, like the computer.
I hope this answers your question, or at least gives you a small appretiation of the task, even if the result isn't a positive one. But people are continually working in this field, and I am sure, that progress will be made.
Sincerely,
Bernd Kohler, bekosoft. |
(Message sent Sat 7 Oct 1995, 03:54:59 GMT, from time zone GMT-0500.) |
|
|
Mechanical
Music Digest™ Archives
