This paper examines whether participating in tax avoidance activities is related to corporate debt policy and what the role of cost of debt is in this relation. Using the tax avoidance measure modified from Desai and Dharmapala (2006), we find the substitution effect of tax avoidance for the use of debt for a large sample of Korean firms, consistent with Graham and Tucker (2006). Further tests show that the substitution effect becomes stronger when the cost of debt is high. These results are robust to a variety of tests for alternative measures of tax avoidance and debt/asset ratios and endogeneity of tax avoidance. The overall results suggest that the tax avoidance as non-debt tax shields (NDTS) substitutes for the use of interest tax deductions and that the tax avoidance activities offer a partial explanation for the underleverage puzzle.