The relaxation time of potential formation across the magnetic field in a collisionless plasma is experimentally studied. The time variation of the local potential is directly measured with a high time resolution technique when a pulsed potential is applied between magnetized coaxial cylindrical double plasmas. It is found that the relaxation times of the potential and density decrease with the applied voltage and the density of the target plasma, and increase with the magnetic field strength and the density of the driver plasma. These results could be understood consistently by the diffusion of ions in the driver plasma to the target plasma.