Some assorted thoughts that might be useful for thinking about questions and answers:
a question is a schema with a blank to be filled in by the answerer after evaluation of the meaning of the question.
shared context is inferred as most questions are underspecified (domain of question, range of answers)
a few types of questions:
narrow down the field within which I have to search either by specifying a point or specifying a partition of the search space
question about specificities of variants: who where when
question about the invariants of a system: what, how
question about the backwards facing causation
question about the forward facing causation
meta questions about question schemas
what do we want a mysteriously powerful answerer to do?
zoom in on optimal points in intractably large search spaces
eg specific experiments to run to most easily invalidate major scientific questions
specify search spaces we don’t know how to parameterize
eg human values
back chain from types of answers to infer taxonomy of questions
an explanation relative to a prediction:
a prediction returns the future state of the system
an explanation returns a more compact than previously held causal expalantion of the system, though it might still not generate sufficiently high resolution predictions
How to detect ontology errors using questions?
is this question malformed? if so, what are some alternative ways of framing the question that could return interesting answers
ie the implied search space of the question was not correct and can either be extended or transformed in this way
types of questions are types of search spaces
questions that change weightings on factors vs describe new factors
recogniziton of the many connection types in the human semantic network
recursive questions move along one dimension as they restrict the space. ie where is that at different spatial resolutions
qualitative and quantitative dimensions along which a query can be moved exchanges information about the implied search space
under vs overspecified questions
closed and open search spaces
navigational questions only work under the one off assumption if the search is stateless
termination guarantees
completeness guarantees
optimality guarantees
meta questions about the methods the system uses to navigate intractable spaces
time vs space vs....?
generating candidates is easy, checking is hard and vice versa
unknown metadata for answers
bias variance trade off
failure mode mapping, do failures imply directionality?
eg does a failure of a candidate change which candidate you go to next (stateful)
how to think about attack surfaces for question answer systems
can this get a human to run arbitrary code by counterfactually cooperating with itself on what step of the process it is on? Can this be tested by going through the whole process with human A then scrambling the steps and running through the same thing with human B and seeing if answers diverge?
what clever ways do hackers widen restricted bit streams?
Some assorted thoughts that might be useful for thinking about questions and answers:
a question is a schema with a blank to be filled in by the answerer after evaluation of the meaning of the question.
shared context is inferred as most questions are underspecified (domain of question, range of answers)
a few types of questions:
narrow down the field within which I have to search either by specifying a point or specifying a partition of the search space
question about specificities of variants: who where when
question about the invariants of a system: what, how
question about the backwards facing causation
question about the forward facing causation
meta questions about question schemas
what do we want a mysteriously powerful answerer to do?
zoom in on optimal points in intractably large search spaces
eg specific experiments to run to most easily invalidate major scientific questions
specify search spaces we don’t know how to parameterize
eg human values
back chain from types of answers to infer taxonomy of questions
an explanation relative to a prediction:
a prediction returns the future state of the system
an explanation returns a more compact than previously held causal expalantion of the system, though it might still not generate sufficiently high resolution predictions
How to detect ontology errors using questions?
is this question malformed? if so, what are some alternative ways of framing the question that could return interesting answers
ie the implied search space of the question was not correct and can either be extended or transformed in this way
types of questions are types of search spaces
questions that change weightings on factors vs describe new factors
recogniziton of the many connection types in the human semantic network
recursive questions move along one dimension as they restrict the space. ie where is that at different spatial resolutions
qualitative and quantitative dimensions along which a query can be moved exchanges information about the implied search space
under vs overspecified questions
closed and open search spaces
navigational questions only work under the one off assumption if the search is stateless
termination guarantees
completeness guarantees
optimality guarantees
meta questions about the methods the system uses to navigate intractable spaces
time vs space vs....?
generating candidates is easy, checking is hard and vice versa
unknown metadata for answers
bias variance trade off
failure mode mapping, do failures imply directionality?
eg does a failure of a candidate change which candidate you go to next (stateful)
how to think about attack surfaces for question answer systems
can this get a human to run arbitrary code by counterfactually cooperating with itself on what step of the process it is on? Can this be tested by going through the whole process with human A then scrambling the steps and running through the same thing with human B and seeing if answers diverge?
what clever ways do hackers widen restricted bit streams?