Banks are charging non-customers more than ever to use their ATMs.

The cost of the average out-of-network ATM withdrawal has reached a new record high of $4.72, according to the latest Bankrate.com study, which has surveyed non-interest and interest accounts and their associated fees for more than 20 years, according to a press release.

This all-in fee, which includes the ATM surcharge — the fee that ATM owners charge non-customers — as well as the penalty that banks charge their own customers to make out-of-network withdrawals, is up 33% over the last decade. 

The average ATM surcharge increased 2% to a new record of $3.09, the 15th consecutive year establishing a new record. The average surcharge has increased in 20 of the past 21 years.

The good news is, the fee charged by the account holder's own bank for using another institution's ATM has decreased 2% from $1.66 to $1.63, moving lower for the second year in a row. In fact, the number of banks and accounts allowing free out-of-network withdrawals is at a record high, although this still represents less than one-third of accounts (32%).

"While large banks have extensive ATM networks, many smaller banks and credit unions belong to nationwide fee-free alliances that may have significantly more ATMs available than even the ATM networks of big banks,” Bankrate.com CFA Greg McBride, said in the release "One other option to withdraw money for free is to get cash-back at the point of sale when using a debit card. Banks don't charge for that and very few merchants do either."

ATM fees vary by region. Houston has the highest average out-of-network ATM fee of the 25 major metro areas examined in the study ($5.58), followed by Atlanta ($5.50), Detroit ($5.27) and Chicago ($5.14), while Los Angeles has the lowest ($4.15).