First find the mass of the air in the balloon minus the mass of the air(Venus atmosphere) displaced by the balloon. The mass of the air displaced is determined by the volume of balloon and the density of the air around the balloon. The apparent mass is derived from the subtraction stated earlier. The force can be determined by the apparent mass of the balloon by multiplying by the gravitational force.(negative apparent mass would lead to an upward force, and positive mass would result in a fall)
I think this is correct.
Basically:
Mi - Mass of air inside balloon
Md - Mass of air displaced by balloon
Ma - Apparent mass of balloon (weight after bouyancy is applied)
Di - Density of air inside balloon
Do - Density of air outside balloon
V - Volume of balloon
g - Force of gravity
F - Force of buoyancy
F = Ma*g
Ma = Mi-Md
Mi = V/Di
Md = V/Do
----OR----
F = (V/Di - V/Do) * g
If you're familiar with programming in Orbiter, the Orbiter API should be able to give you the density of the air around your vessel (the balloon) and the force of gravity at the balloon's location. You can probably find the density of various mixtures of gases by researching on the internet. (wikipedia says the density of Earth atmosphere at sea level is 1.2kg/m^3)
I would correct, a bit. g would be the acceleration of gravity, and F would be the "total" downward force (weight of balloon - buoyant force). If I recall correctly, Archimedes' Principle is that the buoyant force is equal to the weight of fluid (e.g. - atmosphere), displaced (by the balloon, or whatever). And since your last formula seems intended to simplify to F=Wb-Wa, where F is "total" force, Wb is Weight of Balloon and Wa is Weight of Air (Atmosphere, and displaced by the balloon), therefore this formula makes sense to me.
Also, it is good that you have complicated it with consideration of densities, since these will vary with altitude (actually, the mass of the balloon should remain constant, so that Wb=Mb*g, where Mb is the [constant] mass of the balloon, while I'm not sure how Wa would be calculated, with changing altitude and thus air density; perhaps the volume, of the balloon, could be calculated from local atmospheric pressure and temperature, and the Ideal Gas Law [assuming an initially known volume of the balloon, for its initially known pressure and temperature], while the atmospheric density, at a specified altitude, could be calculated from an exponential relationship - although, for Orbiter purposes, I imagine that local atmospheric pressure, temperature and density are available as accessible variables within the program; then, Volume of Balloon,
times Atmospheric Density, times g, would provide Wa, since Volume of Balloon is also volume of displaced atmosphere).
We are both assuming, of course, that g (for a specified planet) is constant - which is reliable, I suppose, for this purpose.