the properties panel is built in a way that the content is shown in a panel (otherwise it would be called properties popup ;)). Changing this behavior would be a fundamental change which is not easy to implement (and maybe you might be better writing your own property editing solution).
Another (much more pragmatic) approach would be to extend the properties panel (see this example) and define (or restrict to) the properties you need. This does not get you a popup but control over the content in the panel.