CodeCube.NET

So now that I’m looking @ the next features for Scurvy Media … the issue of Audio comes up.  Currently, if you add a video to play in your XNA project, it will be silent.  Now, the problem is that the “audio pipeline” is completely separated from the content pipeline. 

Even though technically speaking, the XACT project is added and processed by the content pipeline … the audio is still gated by the requirement of wave forms being added to the XACT UI.  Thus, being able to add audio items programmatically is an issue.

I toyed around with the possibility of extracting the .wav and programmatically creating an xact project from the video importer … but comments on the forums suggested that having more than one AudioEngine instance wouldn’t be smart.  So it looks like in the meantime, I’ll have to simply provide a separate tool to extract the audio from the .avi, and then an API to sync up the audio playing with the video.

If anyone has any ideas, they’d be very welcome

Hehe … poking around on the internet, I actually came across an old post of mine on the old XNA forums from when they were on the MSDN forums.

I believe I had posted this on my blog previously as well, however, since I changed blog engines this post has been lost to the ages, so I figured I’d reprint it since it’s still good info.

After reading this very useful post-mortem and working with XNA and XACT for a bit, I decided to do a bit of a pre-mortem to share what I’ve learned so far.

• You should only have one instance of AudioEngine. You can have as many sound and wave banks loaded into the same engine as the hardware will support, but the only restriction is that they all share the same global settings data. This leads me to deduce that the best practice is to have one XACT project for your entire title.
• XACT is already multithreaded. Apparently, the heavy lifting of playing audio on Xbox happens on hardware thread 4 (the second thread of core #2). Calling AudioEngine.Update on a different thread is not likely to bring any performance gains unless you “have millions of crazy curves and variables and things”.
• Despite my initial confusion, XNA does in fact support streaming wave banks. All you have to do is designate the wave bank’s Type property as Streaming, then call the WaveBank class constructor that takes the offset and packetsize arguments.
  The main issue I have with this is that it is not documented very well. They try to provide guidance on what to set the packetsize argument to, but the guidance is for streaming from a DVD (something XNA doesn’t support quite yet). The documentation should also provide guidance for a title running from the HDD.
  Also, it’s important to note that you must call Update at least once and check the Isprepared property to make sure it’s true before attempting to play any sound that uses one of those waves. Thankfully, this is documented in the streaming constructor topic, but I think there should be a “how to” article on how to stream audio.
• According to the docs, XACT supports in-game auditioning using a special development-only API. I have not seen that these APIs are available in XNA … this would be a pretty cool feature
  The toolchain in Superman Returns: The Videogame had a very similar feature where the console would run a tcp/ip server that would change audio properties at run-time. The tool running on the audio designer’s PC would then send messages when its knobs and switches were manipulated. I wrote the piece that ran on the PC (using C#, winforms, and .NET Remoting) … based on the feedback I got from the audio guys, this really helped them nail the audio mix. So I would love to see similar functionality be available to XNA titles in development.

All in all, I think XNA and XACT does a pretty good job of leading people into a Pit of Success. I’ve still got a lot to learn, but I’m really enjoying audio programming so far. Hopefully that will be reflected in my Dream.Build.Play entry.

Sadly, my dream.build.play entry never got to see the light of day.  It was going to be an audio only game in the spirit of “In The Pit“.  Perhaps someday it will still see the light of day … no pun intended

despite the refactoring of the scurvy media pipeline architecture, trying to add large videos will result in an out of memory exception.  This is because under the covers, they’re using a MemoryStream … so even though *I’m* not using a lot of memory … they still are.

https://connect.microsoft.com/feedback/ViewFeedback.aspx?FeedbackID=323222&SiteID=226

I filed the above bug report … and I admit even in that bug report that my pipeline could just avoid writing so much data and I am planning on improving that … as soon as I can figure out how to write and then set the data texture data using dxt1 compression.

But I figured I’d file the bug report anyways as there could be some other reason to write lots of data to .xnb

If you haven’t already seen it, the Scurvy.Media source code contains a sample project.  It’s not very complex, just a simple example of how to use the API to display a short video using the SpriteBatch.

http://www.codeplex.com/ScurvyMedia/SourceControl/ListDownloadableCommits.aspx

Since you can download the whole repository from the link above, I didn’t bother to make a source code release.  But if you download the latest commit, you’ll see the VideoSample project.

Enjoy!

http://www.codeplex.com/ScurvyMedia

The binary release now supports GS 2.0, and sports a cleaned up pipeline architecture that has a better memory footprint, and a cleaner extensibility model.

Next on the list is audio integration and perhaps a few bugfixes.

Please note: currently, the release is only for x86, I haven’t tested the code on the 360 quite yet. More on that soon.

If you receive this error when calling the update method on the Video class:

The operation was aborted. You may not modify a resource that has been set on a device, or after it has been used within a tiling bracket.

Then this thread might offer some insight into how to avoid it:
http://forums.xna.com/thread/35536.aspx

I got around it by putting this line of code to null out the texture on the video card before the update method gets called on the Video class in the sample project:

GraphicsDevice.Textures[0] = null;
vid.Update();

Finally! the scurvy media video-to-texture library finally works in xna 2.0.  I checked in the changes that show off the new features like the video processor that lets you choose between streaming and in-memory playback mode (in the Branches folder). 

thanks for all the feedback that I’ve gotten so far, and for everyone’s patience I do this on my spare time, so it’s pretty satisfying when I finally get somewhere. 

Now that I finally got it to work, I still need to clean up the code a little bit and merge the branch back into mainline.  Once I do that, I’ll put together a binary release.

hmm, interesting … just figured out that in the latest build (not released, just in source control) of the scurvy media library, any input AVI must not be readonly.  Not the worst bug in the world, but a bug nonetheless.

I created a bug item:

http://www.codeplex.com/ScurvyMedia/WorkItem/View.aspx?WorkItemId=3245