Saturday, March 13, 2010

Replacing $ in a String in Java

I faced an issue in my project recently where we read a password for an admin user first from a file which has a dummy or wrong password written in it. Then I read the password from database and replace the dummy password with the real password that I get from Database. So the code was something very simple like this (this is actually different from originalk).

String myXmlDoc = readALargeXmlAsString();
String realPassword = readFromDatabase();
String password = myXmlDoc.replaceAll("dummyPassword", realPassword);

All on a sudden we found that this replacement started failing after the password of that admin user was changed recently. Here is the exception log.

Exception in thread "main" java.lang.IllegalArgumentException: Illegal group reference
at java.util.regex.Matcher.appendReplacement(Matcher.java:706)
at java.util.regex.Matcher.replaceAll(Matcher.java:806)
at java.lang.String.replaceAll(String.java:2000)
at com.salesforce.test.StringReplaceTest.main(StringReplaceTest.java:34)

So I figured out from the log that it’s actually the replaceAll() pattern matching method that is causing the failure when you have $ in your replacement string (not in the pattern itself). So as per the first code example, if you had a $ character as one of the characters for realPassword, then it would throw the exception above. While the solution was to use a different admin for now who doesn’t have a password containing $ sign, I sat down to investigate it in detail and write a long term solution.
First I reproduced the issue in 3 different ways. Here is a code that will tell you depending on where you are putting the $ sign, the exception stack trace will be different. I give you the code and the compile and run instructions so that you can test it yourself.

package com.salesforce.test;

/**
* To compiple: javac -d . StringReplaceTest.java
* To run: java com.salesforce.test.StringReplaceTest
* or, java com.salesforce.test.StringReplaceTest start
* or, java com.salesforce.test.StringReplaceTest middle
* or, java com.salesforce.test.StringReplaceTest end
* @author ashik
*/
public class StringReplaceTest {
public static void main(String[] args) {
System.out.println("\nStringReplaceTest starts.....\n");
String firstStr = "I am a Java programmer working in USA. Chess is my hobby and here in USA lot of people play chess.";
System.out.println("firstStr before replacing = " + firstStr);
String positionPOfDollarSign = "";
if(args.length > 0) {
positionPOfDollarSign = args[0];
}
String secondStr = "";
if("start".equalsIgnoreCase(positionPOfDollarSign)) {
secondStr = firstStr.replaceAll("USA", "$PUT_A_VALUE123"); // illegal group reference
} else if("middle".equalsIgnoreCase(positionPOfDollarSign)) {
secondStr = firstStr.replaceAll("USA", "PUT_A_VALUE$123"); // String index out of range: 15
} else if("end".equalsIgnoreCase(positionPOfDollarSign)) {
secondStr = firstStr.replaceAll("USA", "PUT_A_VALUE123$"); // java.lang.IndexOutOfBoundsException: No group 1
} else {
secondStr = firstStr.replaceAll("USA", "PUT_A_VALUE123"); // no error
}
System.out.println("\nsecondStr after replacing firstStr = " + secondStr);
System.out.println("\nStringReplaceTest ends.....\n");
}
}

Developing the fix was a little tricky as replacing the $ and \ characters (these 2 are what makes trouble) using regular methods like Spring.split() or StringTokenizer class doesn’t work as those itself can’t process $ correctly. So I had to do my search and replace based on the String.indexOf() and String.substring(). Here is my fix and I would like to know your feedback on this. Please note that Apache StringUtils will be a very good resource to use here instead of trying to write the algorithm yourself.

package com.google.test;

import java.util.StringTokenizer;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * To compiple: javac -d . SfdcReplaceSubstring3.java
 * To run: java com.google.test.SfdcReplaceSubstring3
 *
 * @author ashik
 */
public final class SfdcReplaceSubstring3 {

 // private static String firstStr = "I am a Java programmer and a $Chess$ player working in USA. $Chess$ is my hobby and here in USA lot of people play $Chess$. USA had a great Chess player named Bobby Fischer.";

 private static String firstStr = "Java, Apex, $Chess$ and Oracle - which one do you like? I guess Chess. If not $Chess$ then what else?";

 private static String patternToSearch = "$Chess$";

 private static String[] replacementStrFromDB = { null, "", " ", "PUT_A_VALUE123",
   "PUT#A^VALUE!123?a+b/c>d", "PUT$A$VALUE123", "$PUT_A_VALUE123",
   "PUT_A_$VALUE123", "PUT_A_VALUE123$", "PUT_A_VALUE$123",
   "PUT_A_VALUE123$", "\\$PUT_A_VALUE123", "\\\\$PUT_A_VALUE123",
   "\\PUT_A_VALUE123", "\\\\PUT_A_VALUE123", "\\PUT_A_VALUE$123",
   "\\\\PUT_A_VALUE$123", "\\PUT_A_VALUE123$", "\\\\PUT_A_VALUE123$",
   "$PUT_A_VALUE$123$" };

 public static void main(String[] args) {
  System.out.println("\nfirstStr before replacing = " + firstStr);
  System.out.println("\npatternToSearch = " + patternToSearch);
  // System.out.println("Direct replacement: " +
  // matcher.replaceAll("PUT\\$A\\$VALUE123"));
  for (int i = 0; i < replacementStrFromDB.length; i++)
   try {
    System.out.println("\nExecuted test#"
      + (i + 1)
      + ": "
      + "firstStr after replacing with "
      + replacementStrFromDB[i]
      + " = "
      + replace(firstStr, patternToSearch,
        replacementStrFromDB[i]));
   } catch (Exception e) {
    System.out.println("\nExecuted test#" + (i + 1) + ": "
      + "Exception while replacing firstStr with "
      + replacementStrFromDB[i] + " = " + e.getMessage());
   }
 }

 public static String replace(String text, String searchString,
   String replacement) {
  int start = 0;
  int end = text.indexOf(searchString, start);
  if (end == -1) {
   return text;
  }
  int replLength = searchString.length();
  StringBuilder buf = new StringBuilder();
  while (end != -1) {
   buf.append(text.substring(start, end)).append(replacement);
   start = end + replLength;
   end = text.indexOf(searchString, start);
  }
  buf.append(text.substring(start));
  return buf.toString();
 }

}

1 comment: