[SOLVED] Flac lossless format support ?

Just studied the service manual for the CDJ-2000.  So the hold up to FLAC is either the custom TI DSP is responsible for decoding files and its not re programmable (Can't find any info on the chip so its most likely a custom job for Pioneer). If the DSP does not do the decoding then its the amount of code that can be held on the flash memory.  They use a 64Mb/32Mb chip, when in mirror mode that gives it 4MB of storage 8MB at the most. While there is a chip for every function, meaning very little code needs to be written to make the CDJ work, I do see the hold up.  The space available on the flash memory might be an issue plus converting FLAC API libraries to run on a RISC CPU and integrate to the UI is a considerable amount of work.  I can see why Pioneer might not want to dive in that deep.  Though for the sake of pushing the industry forward I wish they would.  

Unfortunate because the parts used are not forward thinking at all especially with today's FPGA's, but what they used is probably reliable and familiar to the engineers at Pioneer.

Also interesting to note 1)the DAC can run up to 192Khz, 2)there is support for LST chunks for WAV files meaning you can have basic tags for WAV's.  I'm going to try writing these tags and test it out later tonight.

It would be great if the players supported AA3, AC3, FLAC, MP4, MPA, AU, OGG, APE, RAW, WMA, QTA, ALAC ... etc, then nobody would have a reason to complain that their codec of choice wasn't supported.

Except maybe those RealMedia junkies, those jerks.

FLAC is a free codec, as is OGG vorbis.  If the Pioneer CDJ 2000 does not have the processing power to do FLAC to PCM conversion, then buyers are being ripped off.  My phone and my two $50 portable audio players (one Cowon, one Sansa) can both do it just fine and have very good sound quality in the process, even with EQ or various processing features turned on.

I know its a pioneer forum but my Denon MC-6000 controller supports FLAC format, very, very cool. I just hope there will eventually be a cd player that supports it cause all my files now are mostly FLAC. The only reason, i think, why beatport went for the Apple format is for MONEY. It's always about money, there is no money to do with FLAC so it's a no-go. If people complain enough the future hardware will support it i'm sure. But they want people to adopt the Apple format. If you're a popular dj and play everyone, well i guess u have to use the hardware that is at the club and usuallly it's a cdj 2000 or something like that so u have no choice to have your files in the format most of the hardware support, so no flac.

Actually, Sebastien, it's your computer software that's supporting FLAC, not your Denon controller.  Without your computer, the Denon is just a mixer and a bunch of useless plastic.

Oups, sorry you are right, so it's my computer that is decoding the flac not the controller.  Thanks for the up.

This issue with FLAC is more complex than just DSP cycles to process.  A media player optimized for DJs require almost random access to track data.   To utilize FLAC, the player has to effectively unwind the compressed file into a WAV to build the seek tables.  CBR MP3s use regular sized chunks (this is also the reason performance with VBR is spotty at best.)   http://www.rockbox.org/wiki/CodecMemoryUsage shows the comparison of typical memory requirements in the decode process (not including the storage space required for the unwound file.)

So, Pioneer would effectively have to bump up the memory (switch from SRAM to SDRAM) which would in turn require the use of a completely different DSP family just to interface to the new memory type.  This in turn would require a complete rewrite of signal processing code.  There are a number of us who experienced the master tempo results of that when the DSP was upgraded from the ones used in the 1000 family to the one currently used in the 2000 product line.

Having been in the DSP business much longer than I care to admit, my personal take is that (as useful as the devices are) the flexibility, speed and overall development advantages have been eclipsed by wide margin through the use of generic RISC devices such as ARM.  If engineering is reading this board, y'all should seriously consider a shift away from DSP in future product generations.  Scary as it is, the benefits in streamlined development, eased support, feature flexibility, and cost savings through simplified architecture can not be underestimated.

Interesting.  It is very good, at least, that the newest Pioneers, Denon, and Gemini players can play AIFF.  Converting you library from FLAC to AIFF gives you lossless and universal tagging, which is really all us quality hounds are after.  Storage size really isn't the issue.  If WAV had universal tagging, none of this would even be an issue.

@ Pope interesting observation, but if the unwinding of a FLAC and its memory size was an issue, then AAC shouldn't be supported.  However, it is and according to the Codec Memory Usage graph AAC uses even more memory than FLAC.  I do agree that the technology and development process used for the CDJ's is outdated.  

I think lack of support for this format comes down 1 or more issues...

Cost associated with supporting FLAC/ Hardware limitations

the Unknown legal Factor or Unintended Consequences  - For example FLAC is a "free" format yet no major player MS, Apple, Beatport, Pioneer etc etc will touch it.  I think this is a legal strategy.  Even though its "Free"  there is always a way to bleed money once a big pocket book enters the picture.

Competition from laptops and controllers could drastically change the role of the CDJ line, thus why bother fixing the current line.

@Ben - My problem with the conversion process is that the tagging supported with AIFF's isn't really a standard.  Its a neat hack adding ID3 as a chunk.  Yes it works on CDJ's and Traktor but it doesn't work in everything.  The reason I support FLAC is that is was built from the ground up to support metadata and lossless audio and saves storage space which is a plus for those of us with huge libraries.  To me it would be a wast of time converting from a great format like FLAC to AIFF.  Also WAV and AIFF have been around since the 80's and have changed very little.  I think the industry could innovate a little more.  Until then I wait till the next great thing.

"Until then I wait till the next great thing."

Which in my opinion is a couple sub-$100 transport controllers, either a MIDI mixer & a 4-out interface or a conventional mixer & 8-out interface or a USB ASIO mixer, and a computer.  It can all come out for far less than half the price of a single CDJ-2000.  If you're content with MP3 and/or WAVs, there are all-in-one units that will do it well for under $500.  For the premium Pioneer gear goes for, it should loaded with premium capabilities.

@Sean:  I did a little bit more digging and you are correct, AAC is even more memory intensive as FLAC.  I've never used AAC, and find it an odd choice from an implementation perspective over FLAC.  I also researched the license for FLAC and it's about as free as you can get, both commercial and non-commercial use is royalty-free.

This leaves two possibilities:

1) there is not enough firmware space in the CDJ to support the additional codec (while not exactly a decode ram issue, still would be an architectural limitation that should have been addressed)

...just had an amusing thought.  We're using CDJ-2000's and here it is 2013;)

-or-

2) The fact that two apple formats are included, AAC is in and WMA is excluded implies that the rather obvious exclusions might be due to licensing conflicts with either Mixvibes or Apple (itunes.)  Note that Mixvibes' native products support .WAV, .MP3, .M4A, .FLAC, .AIFF, .OGG and .AAC.

@Benjamin:  IMO, that next great thing is a pro-grade Aero (that stand-alone plays SD or thumb media); one with select-able cues, hot cues, and a pair of really nice 2000-like displays that support multi-touch.  That way I have one super-reliable thing with which to gig.  Cost is not the priority for me, sound quality, performance capability, and reliability is, otherwise I wouldn't own 2000's and a DJM;)

Yeah I had looked into it earlier.  I think the problem is the firmware space.  If you look at the size of the firmware downloads and read the spec there isn't much room left.  Enough for bug fixes IMO.

I'm with you on the next best thing being a next gen CDJ that I can plug my 2tb hd in and play FLACs with enhanced functionality.  In another post I suggested this and my idea, in short, was to put the brains of a cell phone in the CDJ.  Tons of power and a solid framework to build features off of.  

My problem with controllers and laptops... Besides the tons of time and issues Ive had making them work.  The last thing I want to deal with when I'm mixing and high as f**ck or whatever ;-) is to deal with buggy laptops/ midi mappings/ excessive wires/ or a cheap malfunctioning controller.  I just want gear I can plug in and it works no questions asked.!

.. and for the time being I'd be happy if Rekordbox added support for *importing* FLACs to Rekordbox collection, and *Rekordbox* automatically converted FLACs to WAVs or AIFFs when you export your tracks from Rekordbox to USB stick.

That would be quite simple addition to Rekordbox, and it'd solve the problem for many people, plus it doesn't require *any* changes to CDJ firmware.

Having observed the combination of computer issues, flaky power adapters and USB cables/ports killing so many otherwise epic sets has turned me off of anything "tethered."  Hell, even some of the big guys don't chance it and set up a redundant pair of players with a canned set and the stage is a pantomime act.  Horrific as it may seem, some don't take any chances when the gravy train is at stake...

and FLAC.  I want FLAC because some of the tools I use do not support AIFF.  I want FLAC because my tracks come that way.  I want FLAC because it takes 40% less download time.  I want FLAC because I can put 40% more music on a given media.  

Curious Question:  Does anyone have AAC tracks?  Do you play them out?

the only AAC tracks I have are ones that are only released on iTunes and I don't think I've ever played one out yet..

All you really have to do is keep your laptop lean, defrag the hard drive (any external drives you use with Pioneers should also get that), and turn the network adapter/internet & battery monitoring off. I find they're stable after that, even Windows. Much more powerful than the anemic guts of some of these overpriced transports. There are also inexpensive combo controllers that can play files directly without a PC. One of my favorites is like that and weighs a lot. Built like a tank. I also have a firewire analog mixer that can be a very high-end ASIO interface one minute (gorgeous sound), and then quickly run sound from all-in-ones or transport controller/players (in player mode) if any problems occur. Any issue, flick a switch and the music keeps going. I also have a 4-out Emu interface that sounds great, but sometimes I'm not so sure about digital mixing buses. Best bass (Alesis stuff, DJM-1000, etc), but everything else seems to suffer. I also over the years sort of preferred the warmer, more organic sound of Denon's top players to Pioneers, but the CDJ 2000 Nexus finally seems to have sound up at that level... not quite the glassy, lifeless quality the older CDJs had. Good tone. Different, but the same tier finally with Denon, who I don't think ever did as much with sponsorships and free gear to DJs & top clubs as Pioneer. To be fair, Denon and Gemini haven't cracked the FLAC nut for their transport players, either. Numark did on the HDX, I believe, though.

Your suggestions have been noted and I thank you for them. At the moment, there are no plans to introduce FLAC support but I will certainly put forward your suggestions for rekordbox to be able to convert the files into a playable format on the players. To be honest, I think this would be low priority given that there are plenty of good software programs out there that will cross convert formats for you such as EZ CD-DA. 

@Gavin: Thanks!

The whole point is that I do NOT want to convert the FLAC files to other format with EZ CA-DA. I want to have the FLAC files only once on my hard disk, in FLAC format, because all the other software I use supports FLAC.

So I'd add/import the FLAC files to Rekordbox collection, and when I want to create an USB stick for playing on CDJ, I export from Rekordbox, and Rekordbox would *then* convert to AIFF/WAV to the USB stick, only when I *export* to USB stick, and the converted file goes only to the USB stick, original file stays in FLAC format on the hard disk.

Does that make sense?

@Pasi > Totally understood. But where do you download your music from that you have them all in FLAC format already?

Juno, lots of groups on Basecamp, Trying to Hound Beatport to get on the bandwagon but they won't.  There are tons of sources for FLAC.  I'm also compressing my old WAV collection to FLAC and manually adding tags and album art.  

@Sean > Thanks for adding to the conversation. I believe what Juno do is provide the option to download the WAV file but they'll convert it to FLAC for you... Right? For me AIFF has been my file format of choice with quite some time now - by and large, we're not limited by download speeds / limits nor in or storage capacity.

I totally understand where you are coming from and I appreciate your input.

I get talking about this is like beating a dead horse.  But yes, Juno tags and converts the WAVs on the fly and it works really well.  So I'll just add my bit on this too.  I have well over 2Tbytes of music.  I couldn't imagine what it would be if it were all uncompressed.  The more music the more expensive it is to manage and backup.  Then you have to spend time converting, analyzing etc etc.  Its easy to just write this request off, but it really IS BETTER than any other format out there and the fact that everyone comes up with excuses not to support it kinda boggles my mind.  Yes storage and bandwidth is excess-able  but it can still be expensive and out of reach for many people.  But I think I'll be done with this conversation and move on with my life.  I hope pioneer changes its mind because then I might think about paying the outrageous prices for their gear.  Until then I'll stick with traktor.

@Sean > I'm just trying to gather as much feedback as possible to pass along to the engineers and get all sides of the argument.

I appreciate that!  What would really be great is a new industry standard.  A music format for DJs, that would contain extensive metadata, cue points, beat grids  with a lossless compression scheme with 24/96khz support.  This way you could buy the track,t load the file in whatever software or device the track is just ready to go.No waiting, analyzing...

Until ^^that happens FLAC is what I use.

Thanks for your help

or even move past PCM and use DSD

DSD is a problematic format. Fine for single-pass A/D archiving, but impossible to do post-production or any processing with in the digital domain beyond trimming (must be temporarily converted to a quasi-PCM intermediary format for more, which is what all the DSD studio boards are internally) and has noise issues in its consumer versions that tend to subtract much of the benefit from it. Considering the linear container decoding ability of AIFF, that seems the best option if these already horrendously-overpriced dedicated transport players don't have sufficient memory or processing to handle FLAC. Using Rekordbox to load from a laptop to such inferior transports and do the conversion to AIFF on-the-fly from FLAC might not be such a bad idea. Certainly it's easier to manage and find tracks using a laptop, at least, even if one does not wish to use the laptop fully for the playback and/or mix bus.

There is DSD wide and DXD. Im just thinking of ways to get pitch bending to sound closer to vinyl in the digital domain. I think digital pitch bends sound artificial now.

@Gavin: I've bought FLACs from Juno Download. Also I've converted my existing CD collection to FLAC.

Also known as PCM short or narrow. It's not DSD. Its multibit PCM reworded in marketing gibberish for a format that can't be processed as-is. Pitch bending sounds just like vinyl to me when you actually compare a CDJ to a flat (moving coil) cartridge through good preamps. If you really don’t believe me, try comparing it on a Numark or (better yet, because they sound better) Denon unit with rotating platter. Sounds the same with keylock off no matter what you're doing.  And that's key.  Now, the nudging sounds bigger & more intrusive and doesn't trail off right to us vinyl heads on a Pioneer CDJ (or most players & controllers) because of the jogs being rate-based. If you use time-based rotation on a non-rotating platter, then it necessitates brief vinyl-style nudging, but is horribly unnatural for most users. Companies that have tried it (Sync, American DJ, Vestax) inevitably end up pissing customers off who are used to the way normal DJ CD Players work.  Rightfully so.  Having the tempo reduce to zero or increase to max when you rotate slowly is a pain for those not used to it.  If you want vinyl-style nudging, you really need a rotating platter or DVS with your TTs.  Simple as that... well, it's not that simple as you know with all that gear & cabling for a DVS.  While it's possible you're also hearing low-pass filtering within a CDJ during bitrate conversion (required when changing pitch by way of bit rate in the DSP prior to the CDJ or DJM's own fixed-rate converters), that's pretty unlikely. And you'd have exactly the same problem with PCM short/narrow a.k.a. DSD wide. You can’t get around the math.  Phono cartridges, capacitance, inductance, and a wide variety of electrical and mechanical factors wreck far more havoc on vinyl sound than what the CDJ is doing when changing tempo without keylock.  And that’s not getting into surface-induced rumble and even acoustic energy hitting the tonearm directly from the monitors, the latter which cannot be mitigated by going mono on the subs and just produces grunge.  We're pretty spoiled with what we've got.