Just wondered whether there are any projects yet fiddling with the binary .xy file format of the OP-XY.
So far I made some very basic research and found drum set / patch generator projects but none with a focus on the xy files.
(I really wonder why hardware companies don’t open their file formats or even firmware if there aren’t any security concerns like with smartphones. Personally I’d be willing to pay double the price for a product like a hardware synthesizer if I can expect a creative community to emerge)
I’m a professional software developer, yet with only little background in binary reverse engineering but I guess I can provide a helping hand in case someone needs some support on an existing project.
In case nothing is there yet - anyone else interested in the binary format (in order to be able to modify or maybe even create projects from scratch programmatically)?
You can assume the format is not open source by default simply because the format may not be stable enough at this point to have other developers rely on it.
There are lots of reasons why I’d like to have it:
pure curiosity (I’m quite interested in technical backgrounds)
fiddling with values in ways which are not supported / intended in the UI
auto-generating tracks by scripts or turning midis into OP-XY projects
most important for me as a software developer: transforming from/to a text format would allow me to track, diff, merge, share, pick, etc. stuff as you can do it with open source software (and which is why every non-textual programming language is doomed)
I haven’t done anything with it, but I think it might be possible to make headway by scripting the MIDI CC stuff to slightly vary a track and keep saving files. All the key presses can be scripted on CC 106 I believe. I haven’t found a way to script turning or pressing knobs though which is limiting.
That might be a basis for reverse engineering, but I really wish they would just release a spec for it.
I know this is an old thread, but I’m hoping to rally more folks around this and see if we can get something working.
I’ve started to using Open AI codex as a thought partner to take the demo project files that ship with the device and reverse engineer them. It’s been a bit challenging, because quite a bit rotates even when you make slight tweaks to the file. But I have made a fair bit of headway and had some early success changing the pitch value of note trigs and getting edited files to open. There is a lot of assumptions that aren’t fully grounded yet thought that I need to work through.
It’s a bit painful, because while there is a tantalizing amount of MIDI automation, it’s difficult to fully automate changes and pulling files off the device without a human to press a few buttons on the device. There used to be support for pressing almost any button on the device with MIDI CC 106. They cut that, but you can regress yourself back to firmware 1.0.21 which supports this still. That said, it’s missing just enough to make it difficult to automate. For example you can trigger record and dispatch arbitrary MIDI note data, but then record is permanently latched–screwing up things you’d want to do afterward. Likewise can trigger MTP mode on the device, but can’t intriguer it.
I worked at reverse engineering this for a while, and while it seemed like I was maybe making some progress ultimately I wasn’t able to get to the point where I could write an arbitrary new file with note trigs that I wanted.
It’s difficult to create files by pure automation, so you have to do it by hand. I created a catalog of about 100 files with single changes made to them. I saved the files and detailed notes on what I was able to find so far up at GitHub - kmorrill/xy-format: OP-XY file format reverse engineering in case anyone wants to pickup the breadcrumb trail.
I wanted to post an update for anyone still curious about this thread or who stumbles on it…
I have been making progress reverse engineering the file format of the OP-XY. It’s extremely buggy still, and I have a ways to go to fully understand what every bit and byte in the file is doing. I have had some early success being able to import .midi files into .xy files where the various instruments in the .midi track map into the new project defaults on the XY and have all their note trigs neatly separated across up to 9 patterns (it can essentialize repeating patterns down to their minimum).
I’ve started to work read/write for songs/scene arrangement and it’s very buggy still. I have also started working with how to read/write step components but it’s very buggy.
Hopefully someday we’ll have a full editor that fits neatly in a web page and can read in an XY project and also support editing. I love my XY and I love working directly with it, but sometimes things are kind of menu divey or hard to grasp without looking at a lot of things.
I’ve been playing with Analog Rytm paired up with the XY lately. The Rytm has been fun because you can fully read/write the project/scene/performance-pad structure from the device remotely. I like making music this way where the menu divey stuff I can just prompt for and play with the knobs I want to while noodling/performing. Code for that is up at GitHub - kmorrill/rytm Little jam example of scenes I whipped up https://www.youtube.com/watch?v=9DmCtHQWRIE
Thank you for starting this work! While I don’t have the technical skills to participate, I am watching this with interest.
i’m working on a similar project to parse the Moog One patch files, but it’s much simpler because they are clear text. My interest was not to modify presets, but to only parse them into a more readable format so that I can compare hundreds of them at once.
anyway, I’m happy to know that someone is working on this project.
I’m working on a physical and software harness to cut power, restore power, wait for boot, turn on MTP, swap in a new file, load up the project, and hit play over MIDI to see if the load worked. While I’m making progress doing it manually it’s very tedious and there are about 10 crashes for every one insight about the file. I tested out some little button pressers today that can be programmatically addressed. They’ll press the M1, M4, Project, COM and Shift buttons on the device to drive the tedious parts. Quick proof of concept https://www.youtube.com/watch?v=dgN_lhQzAtM
If the device was more workable I’d risk opening it up to see if I can intercept the GPIO input coming off the keyboard–but don’t want to risk ruining mine.
I believe I can get it in a good loop where it tries a new file every ~15 seconds, which is about 5k files in a day. Wish me luck
This looks really incredible, thanks for the effort! Currently I have only little time to spend with OP-XY related tasks but I’ll come back soon (maybe even with some news about a OP-XY dedicated controller I’m working on).
Let me know if there are specific things I can help with - I have some (professional) Python development background, and time will come when I have some spare time to spend on OP-XY again!
I’ve already tested some similar stuff on the XY too. I plan to add backup and project editing features to OP-PatchStudio (https://op-patch.studio/) in future.
I see from your repo you’re using Claude Code. Claude helped me reverse engineer (legally) the MIDI SysEx protocol used by the EP devices and create EP-PatchStudio (https://ep-patch.studio/). The AI programming tools are so good as helping figure this out!
I’ve already added a similar feature to the EP tool as a Project Viewer, this will allow projects to be editable soon too, see https://youtu.be/9m6I-lhrKd8. I’ve done some similar testing on the XY and after the launch of the new EP tool this week, I plan to go back and show the OP tool.some love and add this feature also.
Also, on your efforts with pressing buttons, you should be able to do this 100% via MIDI. TE are TERRIBLE about documenting all this stuff, but CC106 with various CC values should be able to help you, see below.
Edit: I had been using the CC106 previously, but didn’t realise TE took this away in recent firmwares. We should email their support and ask for this feature to be reenabled.
Not sure if you’re working on Windows or mac, but as I’m sure you know the MTP is a PITA. Not sure why they used it. When I started the journey last year on integration with the XY it seemed TE had their own customised version, but I added XY support to the main libmtp library so recent versions should work out of the box with it - Issue #311 - Added Teenage Engineering OP-XY by joseph-holland · Pull Request #312 · libmtp/libmtp · GitHub .
Also one idea that might help too is feeding the MIDI CC spreadsheet to Claude Code AND giving Claude eyes. You should be able to connect a webcam to your computer, point it at the XY and get Claude to control it. Ask Claude for the specifics, but using something like OpenCV https://opencv.org/ or even just get it to capture images of the camera output then Claude should be able to see the screen of the XY and then understand what to do next. This should help speed things up considerably.
The only thing I’d not have an answer for in software is the power cycling on crash, but if you positioned the XY on some books and maybe used one of those button pushers and a little bit of sticky tape - tape the pusher down on a flat surface and then raise the XY up so the level of the pusher arm is near the power switch, then tape the end of the arm to the power switch also. It might take some messing, but you should be able to get Claude to also programmatically control the XY power too.
I hope all this helps. I’ll stay close to this too and try to help out in future after EP-PatchStudio launches.
Patch Studio looks cool. A lot of people complain about sample management on OP-XY and I think you could already fairly reliably use what I have to build something that scours all your projects to figure out what samples you need on the device.
Yeah, I have to circle back to it, but last time I checked CC 106 is problematic because it permanently latches shift with no unlatch. I was originally checking because I was going to use it to do fairly sophisticated sequences of key presses to exercise a lot of the OP-XY functionality. I ruled it out, because it gets super buggy quickly. Now I really just want to try to fire up a project and do the trial/error work, so maybe it would work for me.
For power, I think I’m just going to use a simple Zigbee enabled controller for wall outlet control. I tried automating flipping the power switch but I have a poorly built early OP-XY and the power button is extremely sticky compared to my OP-1 Field. It requires a lot of torque to move it, so I have a particularly bad unit to try automating the flipping of the switch Maybe I’ll play with this more and find some kind of robotic elevator like a motorized lab jack. I think if my OP-XY is secured tightly and there was a it would work.
The libmtp update is cool I will check that out, because yeah I expect that to be problematic too.
Instant_Fidelity above mentions a list of CC, but here is another list that might be more complete.
Either way neither lists seems to mention that CC86 lets you select a project from your project list, as long as you prefix your projects with 3 digits, such as 000MyProject. In other words, CC86 = 2 will select project 002FooBar. Maybe that will help you.
Maybe I’ll play with this more and find some kind of robotic elevator like a motorized lab jack. I think if my OP-XY is secured tightly and there was a it would work.