Pastors, elders, and other church leaders are not kings like Saul. However, their authority is similarly limited by what God has revealed. Church leaders may not go beyond what God has spoken, and church leaders who try to impose unbiblical demands on people have erred greatly. All of us should seek never to go beyond the Scriptures, especially those of us who are leaders in the church.
It is not uncommon for people to experience fear with respect to the calling they have been given or a sense of unworthiness for that calling. However, such feelings become sin when they keep us from serving as the Lord has appointed us to serve. If God has called us to a task, it is no mark of humility to let our fear keep us from serving where He has placed us. If God has called us, He will equip us for our calling.