My final compiler will generate some kind of pre-optimised AST, instead of machine-code object files. Then the AST files will be linked together into another AST file, which will be copied to the standard library directory, or converted into machine code executables on the end-user's machine.
However, for the initial compiler I'll (most likely) use LLVM's backend to speed up on development time. This way, in order to have good optimisations between different objects, LLVM's IR is probably the only sane option.
What I'm saying is that LLVM's IR isn't exactly ideal, but it's probably the best of what we have now.
In the context of your example - llvm shines since you can write custom "passes" which allow you to modify the AST... so as you try different optimizations you can snapshot the AST, compare the AST, etc... google has some really nice llvm AST manipulation tools