Sorry about the original post and I missed Ray saying his bit was too short. He also got a mini chuck which I am certain will help with small bits as they can move in the chuck.
I cut a block from a 2x4 and as you can see there are prominent growth rings which can make the drill bit wander. These rings can also make doing a compound cut difficult. When drilling with a small bit, you need to bring it back up often to clear the chips. The picture shows a 1/16" bit and I drilled a hole through the 1-1/2" block.
I use a larger drill bit for my compound cuts as the 1/16” bit is short and can easily break and more difficult to thread a blade through. Even with a longer bit, you still face the issue of wandering and breakage. I typically use a 3/32” bit which is 2-1/4” long and much easier to grip in a chuck and drill through a block of wood. Also, easier to insert a saw blade. Of course the pattern may dictate what size you need.
I have cut a lot of 3D ornaments and making them out of this type of wood is more difficult than others. There are hard spots and then soft spots can break.