Clearly define the higher-level goals of this position.
What is the team like today?
How did the need arise?
Are there still some open questions?
Define 5-10 factors that are needed to perform in the job, then have each interviewer score them. This will help your team evaluate candidates during the interview process.
Creativity / problem solving
Written & verbal communication
Responsibility & integrity
Dev ops experience