AI Patch Creation Tool

Share
#6

"
AI

Creating a deep learning tool that analyzes audio data and converts it into a Yamaha DX7 patch requires several steps. First, you’ll need to gather a large dataset of audio samples and corresponding MIDI files. Then, you can use a convolutional neural network to extract features from the audio data. Afterward, you’ll need to train another neural network to map the extracted features onto the parameters of a Yamaha DX7 patch. Finally, you can use the trained model to convert incoming audio data into a Yamaha DX7 patch in real time. It’s a complex process, but it’s definitely possible with modern machine learning techniques.

…

onverting a complex audio waveform into a 6-operator FM synthesis algorithm involves several steps. First, you’ll need to preprocess the audio signal to remove any unwanted noise or artifacts. Then, you can use a Fourier transform to analyze the audio signal and extract its frequency components. Next, you can use these frequency components to modulate a carrier signal generated by a phase accumulator. Finally, you can use six operators (usually sine waves) to generate the desired output sound. This process is known as Frequency Modulation Synthesis, and it’s commonly used in electronic music production.
"

1 Like
#7

As interesting of an idea as it is to use the overtones of a signal as modulators in an FM patch (I think Toybox’s Atomic Oscillator for Reaktor Blocks does this) that is 100% not how you would reverse engineer an FM patch at all.

The approach (from someone who understands how a neural network works) would be to take a large selection of Vital presets and randomly record some sequences of MIDI notes performing them. You’d match up the audio snippet to the preset (as a list of parameter values) in a series of pairs. Then you have a neural network take the FFT and amplitude curves of the audio snippet (there are a few other “observables” as well that might be useful) as inputs and drive a neural network that outputs vital parameter values, and use gradient descent to drive those values towards the known values of the preset.

Basically exactly what the OP tool does, the trick is just WHICH parameters you expose as outputs for the network to train on.

2 Likes
#8

i thought it was a little bit amusing to ask my locally installed LLM how to do it and it tried to answer. (nous-hermes-13b) but for sure, humans could probably do a much better job of it. there must be a pure math way to reverse some program audio material to something that vaguely resembles it with enough FM oscillators though, using ADSR type envelopes. even if it gets close it would be fun.

#9

it is a great tool! definitely cool that it adds the effects to the preset.

#10

I just looked at some YouTube videos and it looks like the sounds are (newbie alert) opened at the oscillator level, and using a noise oscillator as well, so this opens a lot of flexible possibility to then tweak, if so required, the sound. It also allows the newb some learning opportunity to see how Vital reproduces sounds-- if my interpretations here make any sense to the seasoned Vital user.

(I imagine Matt has already been informed of this app.)

I’ll admit that I had been sitting a little on the fence WRT Vital, but MicroMusic may have just pushed me off it.

I mentioned MicroMusic today at KVR, incidentally (in Instruments section), since there was nothing about it when I looked.

1 Like
#11

The idea is awesome… I was about to try it…

However “gui.exe” at least is flagged as malicious by VT. And I agree, the behavior analysis is suspicious to say the least, just the number of IP addresses contacted, many of which in turn are flagged either malicious or suspicious. Sorry to say, either you are using infected libraries of unknown origin or someone is up to no good… Thinking good of people and the sheer amount of work put into this product, I think it is the former.

Link to analysis results removed as it was reported here as spam. To replicate result, upload “gui.exe” for analysis.

Only because your specific AV thinks all is fine, it may not be. Especially in these AI times where an AI can cook up some almost undetectable malwares just as easily as Vital presets. Always check new stuff with multiple AV tools. There are a number of services available online.

Be safe guys!
/C

1 Like
#12

VirusTotal = Spam/FUD?

While I’ve yet to look into it, completely at least, I find it interesting that circular’s first and only (so far) comment seems to be about sowing FUD (Fear, Uncertainty & Doubt) about an app that is probably threatening to the proprietary software model, such as in terms of its speed of development. And/Or it’s spam for their ‘virustotal’.

The MicroMusic app is from a university apparently, it has a Discord presence and, like Vital, may also be FLOSS (although I’m unsure-- yet-- of the latter). So so far, it seems well-vetted, which is more than we can say so far for ‘circular’.

Has circular provided for the readership here what is supposedly malicious? Like what is the virus code or whatever? Doesn’t look like it. Or who/what their ‘multiple vendors’ are? Doesn’t look like it.

Sure, ‘be safe’, do your homework, but don’t give into FUD or spam. That might not be safe.

#13

I’m sorry you see this as potential spam and the spreading of FUD. I see I do a poor job explaining the threat. I stumbled upon Vital just two days ago as I started to follow an artist using it, and I saw this post about Micro Music. As I’m into AI, I thought this is beyond awesome! However, true to my own nature and routines I check new stuff out and VT has been around for at least 15+ years as a free tool, a service with which I have no affiliation what so ever, equally as much as you are affiliated with vendors of services and products you use.

I do however work in the business of cyber security as a pentester and forensics specialist and have been doing so for 20 years. I see too often too many projects get owned by a third party using direct attacks, infected libraries or the like used by the developer. And this is just increasing. Operating systems are getting harder to hack, so the target is now the less secure products used in the operating system. Just google “Hacked open source projects”. Both Ars Technica and ZDNet has good writeups. You would be mindblown if you knew the intricacies of the kill chains commonly being implemented.

I’m sorry you see me posting the VT link here is spam, but it is part of the analysis result of the “gui.exe” binary, part of Micro Music. Who would I be not to have something to back my concerns? There should be no reason for a local tool to reach out to 10 different IPs of which 5 are flagged as either suspicous or malicious. Looking at the community comments also, some of the IPs are known for spreading malicious content, ranging from adware to trojans.

So if the above is “spreading FUD”, sure I’m guilty as charged. I’m just trying to spread knowledge and healthy level of awareness as we tend to jump head over heels into new things. Just because its FOSS and seems to be well-vetted, it does not automatically mean that the author has full control over hundreds of thousands or even millions of rows of code in all the third party libraries used. When I write stuff, I check the libraries I use because I don’t want to contribute to infecting a client’s network. That’s the least I can do. Call it occupational hazard.

I mean no disrespect and you are completely free to disregard my “preaching”. Just trying to help.
/C

4 Likes
#14

If we click the little pencil icon, we may notice that you edited your comment after my response. Fair enough. It makes it look less like spam and you less identified with the service-in-question.

But still no evidence of what the supposed virus/exploit is, despite your supposed related experience in the field. Do you have a web presence after 20 years? If so, what/where is it?

WRT software-vulnerability, you may find that FLOSS software is not any more vulnerable than closed.

And as we see, FLOSS can have faster development and turnaround times. This may include faster AI integration as well as quicker patches and fixes WRT exploits and bugs.

Furthermore, the University of Waterloo, such that MicroMusic’s devs are affiliated with, seems rather renowned up here in Canada, as well as globally, perhaps especially for engineering, including computer and software.

In any case, and what with your ostensible background, presumably you did the right thing and contacted the dev(s) of MicroMusic to let them know of your alleged discovery, along with its details, yes? Because, as you write, you’re just trying to help.

#15

Well you see, that’s the thing… If you don’t want to be the main attraction in the next episode of amateur hour, you do not ship a dropper with malware in it. Today because of aforementioned reasons, malware never comes with the trusted carrier. The malware never touches disk. It only runs in RAM…Reason? For the very same reason your are questioning me :wink: The common delivery mechanism? Ads!..And similar mechanisms. The IPs mentioned in the analysis, some are just that - Content delivery mechanisms known for providing the second, third or even the fourth stage in the kill chain. So I’m sorry to say, you seldom see a smoking gun when just grabbing the dropper part/trusted software.

You will have to get the execution in progress and grab a RAM dump, which in turn means that in order to do so the easiest way would be to run it in a virtual machine. However the “gui.exe” also seems to have VM detection mechanisms included, which is common in malware in order to change behavior to an innocent one. Well yes, this mechanism is also common on commercial software in order to make reverse engineering harder. But as this is FOSS, the latter would hardly be the reason.

So, 20 years may produce tons of online content, some don’t…I’m more of the latter type. But you will however find some at 0x90.se

I never said that FOSS is more vulnerable than closed source/proprietary software. It all depends on packaging and delivery mechanisms. Side loaded and non-signed packages/apps/programs will always run a greater risk of being compromised, be FOSS or proprietary software. But FOSS projects, especially smaller ones that doesn’t have the big spotlights towards them used in bigger projects that are the end targets are today valuable targets and the authors are not always security savvy following the OWASP Top 10 dev guide. I see no difference today between FOSS and proprietary software when it comes to patching in general. Its different from team to team and company to company.

A slight revelation came to me the other day, and this is interesting from a cultural perspective I think. I realized that I’m somewhat guilty of assumptions. There is a net cultural gap here. Where I’m from, the forums i spend time in, the persons are of less importance. The claims and such are of greater interest. If I drop some concern about something, others usually tear the concern (and usually software connected to said concern) into pieces and get back to me and say “Hmm, there is some substance here to support your concern…” Or “Hey dude! You are full of bull!” and we usually have a good laugh about it. Here I see it is slightly the opposite. Here I am the ostensible antagonist in your eyes and I also sense a tone of dislike throughout your writing. Had it been the other way around, some people in my type of forums would think you are defending the suspected behavior in the software and perhaps would have something to do with it. As I said, this is interesting and you always learn from interacting with people. My guilt is that I’ve spent way too much time with people of my like.

I’m not doubting the devs. I’m doubting the libraries and tools used to build said software. And you are so right! Going directly to the devs is the right and most constructive thing to do. But related to the paragraph above, I perhaps assumed too much. Since devs usually hang around the forums they develop software for, I assumed they would be present here as their software has a very slim area of applicability.

I wish you a good day!
/C

2 Likes
#16

Yup.

So what’s the holdup?

If I’m guilty of anything here, it’s being protective of FLOSS, Vital and apps for it that can take it to another level.

Perhaps some ‘industry insiders’ and/or ‘assorted mountebanks’ are threatened by that and AI in general…

Which could be why some might want to go on some sites to spread FUD. Not saying that’s you. Just sayin’.

#17

It’s a bit strange seeing such an ardent defense of FLOSS in this thread about a Windows-only tool that does not discloses its license or even its authors, and heavily leans on Google to provide a mandatory account service.

By the way only Vital 1.0.6 is open-source (thanks Matt!). Technically Vital 1.5.x is not, yet.

#18

Holdup? Well, as adrienbeau addresses in this thread, the contact details of the authors are not easily available, which is common decency. Nor is there any other documentation attached to it. like at least a license type file.

I would say au contraire mon ami. When it comes to both FOSS and AI, persons, groups, academia, organizations, fortune 500 companies throw themselves onto it as someone has done the job already. And that’s my point. They use the same libraries as other projects do. Just look at the “FreeImage.dll” in the “_internal” directory. Many projects use this particular library, be it FOSS projects or commercial proprietary closed source. Thus I conclude that the authors of MicroMusic has not written it.

A person, group or nation state would nowadays rather compromise such a library instead of the closed source program. This way you get your routines into a signed and trusted package, executed with elevated rights as the user OKs this by clicking “Yes” in the Windows UAC dialog. This tens of thousands rows of code project is after all well used and well vetted…

Being students at a renowned college or uni is not a pretext for legitimacy either, and yet again to be clear, I’m not accusing the authors of MicroMusic of anything. The authors of one of the most devastating worms and DDoS attacks in history (Mirai) were a couple of 21 year old US dorm room college students playing a prank supposedly on some Minecraft servers. A 19 year old Italian student was running one of the bigger banking fraud bot nets in Europe a few years ago. Why? He wanted more money to go to clubs and drink champagne.

Last but not least, I would like to ease your fear that FUD is spread by “industry insiders” in order to contradict and prevent FOSS to gain popularity and users as the industry literally thrives upon FOSS. Why bite the hand that feeds you? Long gone are the days of the old Microsoft sentiment to kill FOSS no matter the cost, on the contrary, they now embrace it. This is why most commercial products nowadays provide a community edition or even open source…They get the majority of the bugs identified, squashed and fixed…for free!!

As for “assorted mountebanks” I fail to see how charlatans like these would win or gain leverage from spreading FUD as they on the contrary thrive upon the opposite type of messaging…Unless it is a question of just spreading havoc and ill-will. A mountebank I would imagine would embrace and develop more intricate schemes using AI rather than working against it. No?

#19

A ‘gish gallop’ of words, along with some jabs (that seem to increasingly belie a different agenda), at MicroMusic’s devs no less, that really say very little in terms of actually going over and talking to/working with its devs and getting better insights and then bringing them back here-- you know, for the community?

So, ya, what’s the holdup?

And some relatively-pointless cherrypicking/hairsplitting by a couple of commenters-- adrien.beau and circular-- who, coincidentally/curiously, both ostensibly signed up within days of each other (and around about the time MicroMusic is mentioned on KVR as its own topic), one of whom mentions the other in the same thread about this free AI software.

Funny that.

Seems the MicroMusic devs are challenging your credibility without having to lift a finger.

Hey, MicroMusic’s free, Vital’s free, and Synplant, for example, that recently came out, isn’t. And there are probably other AI synths/plugins/VST’s on the way.

AI is ‘billed’ as a threat or challenge to many industries, professionals and individuals, so I won’t be surprised if we see odd effects from them, moving forward. Whether the billing actually pans out or how is another matter that I suppose time will tell.

1 Like
#20

This has got to be my second favorite thread on this forum.

1 Like
#21

what’s the most favorite? is it the “are you okay matt?” thread?

#22

The matt thread is really funny though, i just didnt have much involvment.

#23

“This has got to be my second favorite thread on this forum.” ~ SlavaCat

You have good taste.

Ultimately, at least to me, using Vital in the bigger picture is small potatoes, but once in awhile when I have the time, I don’t mind picking up a pea beside my small potatoes, loading it in my pea-shooter and firing it off. Even if it takes more than a few edits (launches).

Matt popping into maybe that thread you mentioned (the ‘Matt’ thread?) and saying something about the thread not being productive and as a result him locking it was kind of funny/cute.
My imagination for fun for that was like some of us as kids arguing in a home garage about a go-cart he made, and he pulls into the driveway after work and tells us that we’re not being productive and to go and get cleaned up and ready for supper and he’s going to close and lock the garage.

:smiley:

Thanks for your coding by the way.

1 Like
#24

Yup circular seems to know what they’re talking about, mighty interesting!

#25

i wish i didn’t need to sign in to download it. can’t remember the password i use for the account i use for these kind of things. i guess that’s a necessary speed bump to stop bots from eating bandwidth though. guess i’m a bot. (how could i even know for sure?)