Electrolytic capacitors are known to be fast devices with very low time constant and able to deliver high power. This class of capacitors is then an interesting technology to power miniaturized embedded electronics for Internet of Things applications. However, the current electrolytic capacitor suffers from its bulky size that does not fit with the miniaturization. To solve this issue, a proof-of-concept consisting of miniaturizing an electrolytic capacitor based on tantalum materials to give rise to a new class of electrolytic micro-capacitors is proposed. To reach this ambitious objective, thin films (<100 nm) of tantalum metal (Ta), tantalum nitride (TaN), and tantalum oxide (Ta2O5) are deposited on a Si substrate by sputtering deposition method. After a careful optimization of the deposition parameters, Ta/Ta2O5 and TaN/Ta2O5 electrodes (Ta and TaN ≈45 nm and Ta2O5 ≈25 nm) and study their behaviors when biased at high voltage (>20 Volts) in aqueous electrolyte are produced. The Ta/Ta2O5 and TaN/Ta2O5 interfaces when the electrode is polarized near and beyond the breakdown voltage of the dielectric layer are carefully investigated. Polarizing the electrodes beyond the breakdown voltage are shown to result in anodization-like mechanisms. In the case of the TaN/Ta2O5 electrode, an N-rich porous layer grew within the Ta2O5 layer as polarization increased. A comparative study on the 2 stacked layers electrodes with different compositions (Ta/Ta2O5 and TaN/Ta2O5) but similar thicknesses (45/25 nm) is carried out: both electrodes show excellent capacitance retention of over 90% over 300 000 cycles. The frequency behavior of Ta/Ta2O5 and TaN/Ta2O5 electrodes shows that both are potential candidates in electrolytic micro-capacitors for powering miniaturized electronics.