Data Model / Mental model
spinmeister |
.
permalink
Thu, May 1, 2008 @ 12:47 PM
This thread is an attempt to create a focal point for a discussion around a mental model (i.e. how people think about snippets of music / sound) and data model (what software developers use to create databases, which underly their software) for the wonderful collection of musical data and meta data that resides here at ccMixter.
I had mentioned something in a different thread and VS encouraged me to start a separate thread on this topic, because it would also give him a good outlet to communicate some of the background thinking and reasoning behind the current version. I’ll seed this thread with several sub conversations in the attempt to create a bit of structure between sub-topics of this rather large one. Why I think, this conversation might be valuable? — imho, finding stuff is still the holy grail of an already very and ever increasingly rich resource like this. To find things, one needs to have some sort of underlying common model for how things are called and how they should be grouped. — and I do believe there’s a feedback mechanism: Stuff that can be found easier will be used more. Stuff that is used more encourages creators to make more stuff available. OK - now I’ll start some individual sub threads, but of course anyone else can start one of those, too. :-) |
spinmeister |
.
permalink
Thu, May 1, 2008 @ 12:54 PM
The concept of a song
In some ways, ccMixter has some similarities with sample libraries, but in other ways it transcends sample libraries and it could become even more so, because ccMixter is a lot about songs. A cappella’s are mostly songs - and there is a reasonably good subset of audio data here, which are parts of songs. This sub thread is there to invite a discussion, if the concept of a “song” could be strengthened in the meta data here. It’s definitely not applicable for everything here, but numerous remix packs, a cappellas, remixes, instrumental audio and midi performances might be linkable to a common “song”. Also, this is where remix vs, remake might be interesting sub discussions. Would it make sense to capture and track the notion of “song” separately from the concept of “sample”? |
.
teru has been asking for a ‘project’ tag since about day 1 but I want to frame this in the context that I laid out below in my meta-response to the thread (read that first): What would be the goal of such a structure?
Let’s say there are 2 main groups of visitors to the site: music makers and music consumers. A project model would mean nothing to the consumer. My assumption is that music makers are almost exclusively interested in digging - I never saw how a project model would facilitates that (?) |
|
.
Some of the reasons why I think having a way to link individual files to a common object like project or song:
Facilitates the notion of a set of samples, which are part of a bigger thing. like a remix pack for a song. We’ve got those all over the place, but now people need to put it into zip files and/or secondary file uploads to keep these things together. I know I’m not the only one who has done that. Besides, I thought what teru says, is law around here ;-) |
|
.
I think I’m missing but that won’t stop me from taking a go at it ;)
- I still don’t see how ccM presenting the notion of a meta-project will help people find samples for a remix they are in the middle of working on. - What’s wrong with uploading a zip or adding files to a record? And how is what you’re proposing different/better than what’s happening now? (zip file stuff will be addressed below) - Unless we have a very strict, one-path UI, some people will make up their own notion of project and (I promise) create a new ‘project’ for each drum part to the same beat (project 1: “My Hi Hat for Foo Session I did last year”, project 2: “My Kick for Foo Session I did last year”, etc.) Have you noticed the new ‘Submission Succeeded’ page? Half of it is taken up with BIG HUGE button to ‘Add Files’ to the record you just created. Once a week, somebody goes out of their way to click away from this screen, back to ‘Submit Files’, then ‘Submit Samples’, then fills out the entire form again and again and again with description, tags, bpm, license etc. for each part. I obviously haven’t figured a way to encourage people to package like samples into one upload record without hugely draconian policies. |
|
.
actually I think we are talking about the same objective. But I may have inadvertently confused it more.
The problem: uploads which are in a zip file and/or “additional files” are can NOT be found and/or auditioned individually. The become kind of lost to the site. This invisibility discourages people from using the zip files and “additional files” type of uploads. So if I have a guitar track in my zip file or as an “additional file”, it can’t be auditioned by people looking for guitar tracks. So the uploading isnt’ really the problem - the finding and auditioning is. Does that make any better sense? |
|
.
Spinny, I think you’ve nailed the point best here, which you’ve referred to in other ways and other places. We can’t audition any but the main file directly (w/o dl’ing the files and auditioning them separately). Fourstones, I think you’ve also brought up a specific situation that goes perhaps without full understanding in some cases, and lost in post-context to others or just plain ignored. Example: I upload a file and don’t have any subsequent files with it at the time I upload. When I come back later to requests for stems or just add other files such as stems, I either don’t think back to using manage files (which I certainly found my way to through viewing posts and happenstance) and add the files or perhaps I figure if I just add these files then without further notice no one will notice they’ve been added. While it’s easy enough to post an announcement, it’s much easier, though less orderly, to just go ahead, click through the tabs and add a new sample. I think this may be the general situation out there.
This isn’t to say that cc doesn’t work great as it is but could still work better, it just is a bit of human nature and first learned behaviors. When I came here first, I know that I learned first how to upload a new sample. The submit files tab is always visible, therefore, out of site, out of mind. I have to click directly to a file to manage it, and that takes thinking. I no lazy bum when thinking, but it took me a while to get around to managing files and even longer to understand replacing (which was just a couple of weeks because I was here most every day for that time). Hopefully that wasn’t too much of an explanation. Getting to changes… well, I’m kind of in agreement with fourstones on the project tab, however, the point about breaking up each user’s files based on type, such as is done on the site, and was mentioned elsewhere in this general topic, definitely appeals to me. As for using my DAW to audition material, I’ve only recently come around to the reality that I could do this (sounds ridiculous I know, but I’m still very young at learning the software and the process). Mostly though, I do go around listening to samples one at a time, which takes lots of time, then download, then eventually sample. Mostly I build a base of samples I might choose from. I work the same as a painter, load my palette with the generals I feel I’ll use, then mix up as I go, only occassionally adding hues and colors to the palette if necessary. That’s for what it’s worth. |
|
.
great feedback (!) there’s a lot of low hanging fruit here (menu items, sidebar trinkets) that should make stuff a little more obvious but unfortunately I just found out I have a new album available and need to focus on that, just a little…
|
|
.
just sampling so far, everything sounds great, but I love track 2! I guess, I can’t give it an ed. pick, so this will have to do: both way up!
|
|
.
OK, so the last couple of days I’ve been taking a serious look at this, prototyping user interface and quite frankly I could not come up with any serious changes that would solve these issues in a way that didn’t confuse things even more.
I do have an idea to make the ‘Add Files’ and ‘Replace’ functionality a lot obvious and that’s to plant it into the ‘Submit Files’ page. I won’t be able to update the site for the next few days but I think we’re ok there. Wish I had better news…. VS |
|
.
were you still considering adding a selector to the uploading of secondary files?
e.g. have a file of type “sample” as a secondary file to a primary file of type “remix” (and any other combination as they case may be). |
|
.
|
|
.
very cool - I’m looking forward to that - I think that represents a significant enhancement along with the recent a cappella and sample browser/search enhancements you have already implemented.
|
|
.
ok, I’ve got the beta site set up and running - anybody willing to test out the new features let me know.
|
|
.
since I’m the one who started this whole thread, I better volunteer to test :-)
you have my coordinates - feel free to send testing url and details |
|
spinmeister |
.
permalink
Thu, May 1, 2008 @ 1:08 PM
Making the tabs of an artist pager more similar to the site wide tabs?
ccMixter contains a variety of musical “stuff”. Musical pieces, sounds, finished works, playlists, reviews, recommendations and discussions. Would it make sense to make the individual artist tabs follow a more similar high level structure to the main site? Every user (not only artists) is a little publisher of “stuff” which is a subset of the whole site. So if it makes sense to make high level (via the tabs) distinctions between Remixes, Samples and A Cappella’s at site level, wouldn’t it make some sense to have the same kind of structure for each site user? Of course some tabs would not exist for some users, but that doesn’t invalidate the idea. Being one of the publishers at the site here, I would prefer to have a tab featuring my more complete musical works (remixes), and separate tabs for musical components I contribute. And depending on what I’m doing at the time, I would like to have a page (with it’s convenient play all feature) where I can listen to all the remixes of a specific remixer. Other times I’d like to peruse just the musical snippets of one of the participants here. I’m full aware that this can be done with a few extra clicks already. But that’s exactly the point: It would be nice to have single clicks for the most common activities and multiple clicks for the more rare one’s. But maybe I’m alone in the way I use the site, so I’m curious what other may think. |
.
Here’s something I’ve been thinking about for a long time but never worked out the best way to present it:
Imagine that you could have up to 3 playlists that make up your ‘ccM home page’. The system default would be all your uploads, sorted by latest. But ‘power’ users could make these playlists anything they want, for example - My highest rated uploads - My Latest Remixes - My Pells Since these are dynamic you could invent your own tagging policy on your uploads, say ‘finished’ or ‘needs_work’ then create playlists that reflect that on your home page: - Finished Remixes - Needs Work (Feedback requested) These are examples of dynamic playlists, but you can author ‘static’ playlists as well and order them how you like. Again, it’s mainly been a matter of presentation - every time you create or edit a playlist properties I guess you’d have options like “Put this on my home page []” with ordering stuff. Powerful, but only once you figure it all out - which is not a trivial matter. |
|
.
I think you maybe talking about running, while I’m still talking about walking. :-)
My primary question was: Why not the same mental/data model for each user as for the site at large? If the model is universal enough for the site, why not for every user as a starting point? Fabulous idea about customizable “home page” for users, but it strikes me as being a luxury on top of the common mental/data model question. |
|
.
point taken, I’ve seen this other sites (tabs for ‘loops’, ‘pells’, ‘tracks’, etc.) maybe I’ll do something like that, have dynamic tabs spring up if someone has uploaded more than one kind of submission - but I have to admit it’s kind of low priority for me personally because this only applies to less than 2% of our users.
|
|
.
|
|
spinmeister |
.
permalink
Thu, May 1, 2008 @ 1:14 PM
The convenience of uploading vs. the convenience of finding/downloading
Would it make sense / is it technically reasonable to allow zip archives to be uploaded, but then to have them extracted on the server and and automatically listed as separate files, so they can be individually searched, previewed and downloaded? Similarly, might it make sense to have individual listings for the secondary files, too? The stuff inside zip files and the secondary files is relatively hard to find, evaluate and grab. |
.
Quote: spinmeisterzip archives to be uploaded, but then to have them extracted on the server and and automatically listed as separate files, so they can be individually searched, previewed and downloaded?
There’s a lot of little pieces to this but it’s doable: - extraction from zip is the easy part - if we offer a direct link to an ‘inner’ file extracted from a zip, that means we must verify the file type for security reasons. I’m going to insist on that because we don’t ever want to be the site that distributed some trojan malware - what to do when one of the files does not verify is whole set of policy conditions we have to figure out. - I’m a little worried about having it so easy to start streaming uncompressed WAV and aiff files; that could break the Internet - a whole new level of file management has to be implemented for when the uploader replaces or deletes the zip - whatever else I haven’t thought of that will become painfully obvious once we 80% into coding it again, all doable but it’s a relatively major undertaking that would affect the stability of the code (as it is). Quote: spinmeisterThe stuff inside zip files and the secondary files is relatively hard to find, evaluate and grab. really? In the sample browser? It lists everything I can find with two button: listen and download (??) I’m not sure what else I can do. |
|
.
so am I uploading things wrong?
For example, have a look at: the most recent upload I did for my “a minor theory” project - a complete remix pack for a song called “Wait” including the a cappella tracks and the instrumental tracks (it’s from the same song after all); it’s here. It has “additional files”, a couple of which are zip archives. One of the existing tags is: male_vocals. But I don’t seem to find any of those files (either the “additional files” or the stuff inside the zip archives) in the sample browser when searching for the “male_vocals” tag. How should I upload such a complete remix pack for a song, such that: * the parts are findable individually (and ideally can be auditioned, if they are mp3) * the parts are identifiable as belonging to the same song |
|
.
ah yes, well that is a hole - we don’t have a way for the user to cross-submit across submission types. iow an upload record is one of pell, remix or sample. The admins can do it (and I just for your upload). To solve this I can add a drop down the ‘add files’ screen to pick the submission type - potential abuse notwithstanding.
For zips what I can do is (at least) expand the list of contents in the samples browser - other zip issues spelled out above. VS |
|
.
Being able to assign “sample” or “a cappella” to an “additional file” will definitely help. I assume you will make that assignable to existing “additional files”, too, so people can retroactively correct some of their uploads? How do midi files fit into that high level tree? Are they supposed to be in the “samples” section? Also the implication is, that one definitely needs to keep “a cappella” and “sample” type files in different zip files at the very least. So that’s something I may have to correct for some older uploads.
The zip file issue may still be messy, because zip files can contain a wide variety of differently taggable items. In my example, there are guitars loops, drum loops, bass parts, even a phrase of spoken word in the same zip file. So to make those findable in the sample browser, I would have to attach all applicable tags to the zip file. Might lead to some pollution of the sample browser results, but it may be the lesser evil. And of course, one still can’t preview the samples inside the zip file, so samples inside zip files remain the poor cousins. And that in turn encourages people to avoid zip files, if they are eager to have their samples used. not an easy one to solve … :-( |
|
.
Quote: spinmeisterI assume you will make that assignable to existing “additional files”, too, so people can retroactively correct some of their uploads?
Nah, there’s only 26 total remixes that also have zip files, I can just run a query that blasts the ‘sample’ tag onto the record. Quote: spinmeisterSo to make those findable in the sample browser, I would have to attach all applicable tags to the zip file. Might lead to some pollution of the sample browser results, but it may be the lesser evil. Yea, exactly - should be slightly less evil if I show you the contents of the zip |
|
.
|
|
.
|
|
.
very, very cool - both the zip file listing and the streaming preview. Thanks!
|
|
spinmeister |
.
permalink
Thu, May 1, 2008 @ 1:28 PM
instrument groupings
this thread is an attempt to come to a common understanding of sensible instrument group names. I’ll just start and hopefully others will continue and/or argue :-) Piano Guitar Bass Percussion (atonal) Tuned Percussion Strings Brass Woodwinds Synth Plucked Bells (or is that tonal percussion?) Vocal SoundFX Animals Machines Nature Human Activity A separate data item might be if the sound is heavily processed or relatively natural) - obviously a lot of gray area there, but in many cases it may make sense to track that? |
.
a good list, here’s another try:
Piano Guitar Bass Tuned Percussion Strings Brass Woodwinds Synth Synth Plucked Bells Vocal Drums Kicks Cymbals Glockenspiel Organ SoundFX Birdsong Animal (non-avian) Industrial Wind Water People Drones Atmosphere (industrial) Electro-beats Glitch fx |
|
spinmeister |
.
permalink
Thu, May 1, 2008 @ 1:35 PM
additional meta data items?
- key signature (for a melody this is testy but for chord oriented material it might make some sense?) - scale (major minor, pentatonic, mixolydian, etc) - processed (clean vs. reverb, delay, overdrive, others) larger text fields (optional of course): - lyrics - chord progression |
spinmeister |
.
permalink
Thu, May 1, 2008 @ 1:41 PM
separate high level section for midi files?
I’m not sure, if I’m alone in this, but I would really love to see much more midi file activity. Midi files allow a different and often additional levels of production and manipulation than audio files. Midi files are tiny! |
.
|
|
.
could that be a chicken vs egg problem? no infrastructure for midi, therefore no activity?
Maybe some evangelism needed? Maybe I’ll conduct an experimental midi collaboration project here and/or do some pluggy plugging to see if interest shakes out or not. |
|
.
Start evangelizing. I haven’t messed with midi much yet, but that’s partly because I’ve been starting out by getting quite into straight mash-ups using 95 to 100 percent samples and modifying them as little as possible to create my mixes. I would be game for working with midi. It may take something of a push for people to work with midi, because it is so much harder to envision what it could be beyond the plunky midi sounds you’d audition the files with.
|
|
.
|
|
.
lol - don’t worry, I won’t stop by your front door with midi manual in hand trying to convince you of the path to enlightenment. :-)
Just thinking of maybe finding out if there are enough of us midi heads around here to add meaningful midi goodness to the current audio goodness. :-) But you’re right, it doesn’t make sense to start writing/changing code unless there’s a proven midi sub-culture. |
|
.
In case anyone is looking for MIDI files. I’ve had my eye on these for a while.
Wierd Polymer |
|
.
cool - thanks! also TheJoe uploaded his lovely piano track to his latest remix as an “additional file”.
|
|
.
|
|
.
maybe we have to sort of a midi blitz (not to be mistaken with a ballroom blitz) or midi group remix project? If you start a separate forum discussion thread on that, I will gladly join that conversation - let’s see if there are any other people who might be interested in the midi side of the house.
|
|
victor |
.
permalink
Thu, May 1, 2008 @ 3:55 PM
some now, more later…
So the prime objective is productivity and I’ve always thought that means you’ve found a pell and you want to dig for samples to fill out the arrangement. (If there are other top shelf scenarios now is time to be heard.) I would venture that nearly 100% of our artists use their DAW software to browse sample libraries, probably while some skeleton of an arrangement is looping in the background. I spent a fair amount of energy a few years ago trying to convince software makers to create a remote browser with open protocols like our query/sample API because quite frankly, that would trump anything we do here at the site. Imagine sitting in Sonar, FL, ACID, etc. with the ability to browse our (or freesound) directly, preview, then double-click to download the sample and put it directly into a track in your project. Alas, no takers. I’ve toyed with the idea of subverting the VST API to build one myself but never quite had the energy. I think whatever we do on the site has to fit that work flow (again, subject to others convincing me of important alternate scenarios) but barring that I’ve assumed the best thing we can do is stay the heck out of way. iow we should impose a particular user model with great care because our artists already have a user model as dictated by the DAW software they use - not that site couldn’t use a lot of improvement, I just want to make sure we are following the same basic principles and goals - once that’s established, it becomes much easier to make individual decisions. VS |
.
I hear you loud and clear and am a big fan of following industry standards, too. However in the area of sample/sound/snippet meta data, there is no industry standard that I’m aware of.
So it’s everyone for themselves and $deity for us all :-) If some of the software players had a bit more vision and nerve, they would pursue your suggestion with a bit more enthusiasm. See what midi and VST have done . I think it made money for everyone. |
|
.
there are several ‘popular’ file formats that would be good candidates (ACID and rx2 come to mind) - but I was making a point about the line between music production work flow and the ccM user model. Our interface should work as a part of the process of making a remix, not a separate model that we impose where you were working on remix but then you had to stop what you were doing to deal with ccM and then, once done with the site, go back to remixing.
regarding file formats: until and unless we start really building out the back end to do the work of converting each upload to the various popular formats (e.g. you upload an MP3, we convert that to ACID WAV) I think we are stuck with the loosey-goosey thing we have today and wait for software DAWs as they import more formats. |
|
.
to me the difficulty lies in the additional information of acid wav files and rx2 files. That’s an entirely different level of complexity than converting between ogg and flac. I seriously doubt that acidicing and rx2’ing can be automated. It (still) requires human musical judgment.
Or put in other words: a one slice acidiced wav file or rx2 file is no better than a regular wav file. A badly sliced one is probably worse. |
|
.
|
|
.
I read a very interesting thing recently to do with just this concept of lack of industry standardisation re: file formats at the moment.
I can hardly remember it now, but I remember the conclusion, its all down to commerce. Companies at the moment feel they have to protect their sample banks as much as possible, so they only venture out into the ‘common’ fiel formats as much as they feel they have to to make their software viable. |
|
.
Quote: fourstonessome now, more later…
So the prime objective is productivity and I’ve always thought that means you’ve found a pell and you want to dig for samples to fill out the arrangement. (If there are other top shelf scenarios now is time to be heard.) Agreed. However, in the lingo of interaction design, there are other personas – the music consumers – that could be better catered to, IMHO. Quote: fourstonesI would venture that nearly 100% of our artists use their DAW software to browse sample libraries, probably while some skeleton of an arrangement is looping in the background. I spent a fair amount of energy a few years ago trying to convince software makers to create a remote browser with open protocols like our query/sample API because quite frankly, that would trump anything we do here at the site. Imagine sitting in Sonar, FL, ACID, etc. with the ability to browse our (or freesound) directly, preview, then double-click to download the sample and put it directly into a track in your project. Alas, no takers. I’ve toyed with the idea of subverting the VST API to build one myself but never quite had the energy. Yes! I have wanted something like this for some time. There is a ccmixter API?! Is this SOAP-based? How do I learn more about this? Quote: fourstonesI think whatever we do on the site has to fit that work flow (again, subject to others convincing me of important alternate scenarios) but barring that I’ve assumed the best thing we can do is stay the heck out of way. iow we should impose a particular user model with great care because our artists already have a user model as dictated by the DAW software they use - not that site couldn’t use a lot of improvement, I just want to make sure we are following the same basic principles and goals - once that’s established, it becomes much easier to make individual decisions. VS In strong agreement that the users (some set of personas, see http://www.cooper.com/) have internal models that are already being used. Anything that creates friction with that could greatly impair the usability of ccmixter. |
|
.
Quote: vo1k1There is a ccmixter API?! Is this SOAP-based? How do I learn more about this?
It’s RESTful. About 15% out of date http://ccmixter.org/query Regarding music consumers that’s another topic, this one is about samples. VS |
|
.
|