I want specific control over the users who can use the add-in
You can do so in the registry.
By default the add-in will be loaded for all users. However, this behavior can be changed easily by removing the add-in from the HKLM and adding it to the HKEY_CURRENT_USER.
https://www.dragdrop.com/download/DragDrop_UserInstall_x86.reg (use with 32bit Outlook)
https://www.dragdrop.com/download/DragDrop_UserInstall_x64.reg (use with 64bit Outlook)
The x86 and x64 are linked to your OFFICE version (NOT to Windows!)
Please note that the removal of the keys from the HKEY_LOCAL_MACHINE might require admin/installation permissions.
Windows Registry Editor Version 5.00
[-HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\Outlook\Addins\DragDrop] [-HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Office\Outlook\Addins\DragDrop] [HKEY_CURRENT_USER\Software\Microsoft\Office\Outlook\Addins\DragDrop] "RequireShutdownNotification"=dword:00000001 "LoadBehavior"=dword:00000003 "FriendlyName"="DragDrop for Outlook" "Manifest"="C:\\Program Files (x86)\\DragDrop\\DragDrop for Outlook - Enterprise\\x64\\DragDrop.vsto|vstolocal" "Description"="Enables you to drag emails or attachments to your browser (Current User mode)"