When someone claims to be a skeptic, the word can have various meanings. Absolute Skepticism is a philosophy that nothing can be known except for the knowledge that nothing can be known. So how do you know even that is true?
More commonly, people are skeptical of news or claims. Healthy skepticism demands support or verification of claims and helps people from being gullible. When it comes to atheists who claim to be skeptics, they usually make it clear that they cannot be convinced about God.
Atheists often ridicule God, the Bible, Christians, and so on. These should be considered scoffers and misotheists, since a skeptic (in common usage) will consider evidence. On occasion, an unbeliever will want to have a discussion about evidence for the existence of God.
Christians often argue in the worst possible way: Letting the unbeliever dictate the terms of discussion, and when asked to argue "on neutral ground," they are actually admitting that the Bible is wrong! Learn this well about when people ask you to be neutral on these things: They aren't, and you shouldn't be. Also learn that neutrality is a myth.
Proverbs 26:4-5 is not a case of a biblical contradiction (the author put them right next to each other). What we have here is something that is called the Don't Answer/Answer Strategy (or similar names). It is powerful and will help us in our apologetics and other discussions when applied properly.
Skeptics of the Bible often grasp at straws, trying to find a contradiction in the Bible so that they can justify dismissing the Bible as what it claims to be—the Word of God. One place they often point to is Proverbs 26:4–5, “Answer not a fool according to his folly, lest you be like him yourself. Answer a fool according to his folly, lest he be wise in his own eyes.” A superficial reading of these verses suggests a contradiction, but ironically, these verses actually provide a profound strategy for answering the arguments of skeptics.
It's in your best interest to read the rest over at "Answering the Skeptics." The following video restates some material in that article, but also add some important information.