Last question first, yes float: right is allowed in HTML5. Now as to the other issue - mp3 will not play in all browsers (add an alternative .ogg version source), and autoplay will not execute in most hand helds (I don't think you can have your cake and east it too on this one). For autoplay what I'm saying is that if you choose autoplay, a custom button is going to get messy in browsers that don't allow autoplay, whereas the default controls will take into account that autoplay didn't fire. Or you can just assume that all hand helds will not do autoplay and make their tags not include that.
With those things in mind you can have a custom button something like so:
<!DOCTYPE html>
<html>
<head>
<title>Custom Audio Control Demo</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css">
audio {visibility: hidden; height: 0; width: 0; position: absolute; top: -300px; left: -300px;}
#pauseplay {float: right; width: 64px; height: 64px; overflow: hidden;}
</style>
</head>
<body>
Hi!
<audio controls loop>
<source src="ocean-wave-2.ogg" type="audio/ogg">
<source src="ocean-wave-2.mp3" type="audio/mpeg">
</audio>
<img id="pauseplay" src="button-play_blue.png" alt="button" title="Play/Pause">
<script type="text/javascript">
(function(){
var playing = !!('ontouchstart' in window) || !!('ontouchstart' in document.documentElement) || !!window.ontouchstart || (!!window.Touch && !!window.Touch.length) || !!window.onmsgesturechange || (window.DocumentTouch && window.document instanceof window.DocumentTouch),
snd = document.getElementsByTagName('audio')[0],
ctrl = document.getElementById('pauseplay');
playing = !playing;
ctrl.title = playing? 'Pause' : 'Play';
if(playing){snd.autoplay = true;}
ctrl.addEventListener('click', function(){
if(playing){
snd.pause();
} else {
snd.play();
}
playing = !playing;
ctrl.title = playing? 'Pause' : 'Play';
}, false);
})();
</script>
</body>
</html>
Demo:
http://home.comcast.net/~jscheuer1/side/demos/tidbits/wave/demo.htm
With a little more work you can have the image change to reflect the playing/paused state of the sound.